Database performance in Web applications

by Stéphane

29 10 2008

It’s more efficient to connect a Web application with an Unix Domain
Socket than TCP/IP one (reduced overhead) so I’ll explain the required
configuration with the following pairs:
1 - TurboGears/SQLAlchemy
2 - Django/PostgreSQL
3 - Django/MySQL

1 - TurboGears/SA

SQLObject is dead, isn’t it? So With SQLalchemy, the syntax is:

sqlalchemy.dburi="postgres:///dbname?user=mydbuser&password=XXXXXX" ([1])
http://docs.turbogears.org/1.0/DatabasePostgres

2 - Django/PostgreSQL

You just need to define DATABASE_ENGINE = ‘postgresql_psycopg2′ and DATABASE_NAME. Leave DATABASE_HOST setting empty to use UDS.

3 - Django/MySQL

Create a database in UTF8, either with default-character-set = utf8
under [mysqld] section in the my.cnf file or with an explicit ‘create
database bla charset=utf8;’

In settings.py:


DATABASE_HOST = '/var/run/mysqld/mysqld.sock'
DATABASE_OPTIONS = {
'read_default_file': '/etc/mysql/my.cnf',
'init_command': 'SET storage_engine=INNODB'
}

A - Note about PostgreSQL

When the user isn’t the same one who runs the process, you must edit the PostgreSQL configuration (/etc/postgresql/8.3/main/pg_hba.conf):
# "local" is for Unix domain socket connections only
local   user     database      md5
local   all      all           ident sameuser

You must create an user with a encrypted password (encrypted by default).

$ CREATEUSER username
$ psql
postgres=# ALTER USER username WITH ENCRYPTED PASSWORD 'my_password';

If you want to be sure, remove the lines with ‘host’ to deny nonlocal connections.



[OFF] Et si j’étais un prof de philo ?

by pounard

19 10 2008

.. je pense que je donnerais comme sujet de dissertation des citations chinoises !

Et oui, en balladant sur le net, j'en ai repéré une ou deux que je trouve tout à fait juste, mais libre à chacun d'avoir son opinion sur le sujet.

Petite note, elles sont toutes de Cofucius (étonnant tiens!).

en lire plus


Vertimus 1.0.2

by Stéphane

22 06 2008

Bugfixes and new translations.

To upgrade from previous version, run the following scripts:

  1. sql/upgrade-sql-vtm-1.0.1-to-1.0.2.sql
  2. sql/upgrade-sql-vtm-1.0.1-to-1.0.2.php

* MAJOR FIX to get the last user who has locked the module
* New $url_hosted_link and $url_hosted_name variables to set in
localconfig.inc.php
* FIX Sending of emails to the committers on ‘Ready to commit’
* New Brazilian Portuguese translation by Leonardo Ferreira Fontenelle
* New Czech translation by Lucas Lommer (#237486)
* FIX #237472 reported by Lucas Lommer
VARCHAR > 255 needs MySQL 5.0.3 or later



libmodbus 2.0.0 “Slaves to Our Machines” is out!

by Stéphane

18 05 2008

libmodbus is library to send/receive data to/from ModBus devices which respect the protocol established by Modicon.

This new release contains major enhancements like the slave component.
A special thanks to Todd Denniston for this release.

The Ubuntu users can obtain the packages of this project here:
https://launchpad.net/~sra/+archive

libmodbus 2.0.0 (2008-05-18)
============================
- Slave API
https://blueprints.launchpad.net/libmodbus/+spec/slave-api
- No more glib dependency
https://blueprints.launchpad.net/libmodbus/+spec/glib-dependency
- Unit testing and many test programs
- Waf build support
https://blueprints.launchpad.net/libmodbus/+spec/waf-support
- MacOS X support by Matthew Butch
https://blueprints.launchpad.net/libmodbus/+spec/macosx-support
- No more glib dependency
https://blueprints.launchpad.net/libmodbus/+spec/glib-dependency
- Unit testing (unit-test-slave and unit-test-master)
- Port number is now defined at initialisation by Dirk Reusch
- Better memory management (uint8_t *data and packing of
modbus_param_t)
- Better error management
- Declare many static functions and const arrays
- Enhance an integer division
- The GNU licences LGPL and GPL are in version 3
- Debian and RPM packages (#224496)
- Many cleanups
- Fix #159443 reported by Stefan Bisanz
Index of incoming data in force multiple coils function
- Fix #161989 reported by Konstantinos Togias
Serial device paths more than 10 chars long (eg. /dev/ttyUSB0) don’t
fit to modbus_param_t -> device char[11] var.
- Fix #188189 reported by Chris Hellyar
Compute_response_size() no entry for read_input_status()
- Fix #191039 reported by Todd Denniston
modbus.h is not installed at prefix.
- Fix #211460 reported by Todd Denniston
With TCP, automatic reconnect on error may not be desired.
- Fix #224485 reported by Todd Denniston
libmodbus does not link with c++ code.
- Fix #224496 reported by Todd Denniston
It is easier to install on rpm based systems with a spec file.



Vertimus 1.0

by Stéphane

22 03 2008

GNOME translation teams,
Vertimus is the perfect tool to follow each translation, translate, proofread and enhance the quality of contributions.

https://launchpad.net/vertimus

===================================
Overview of changes in Vertimus 1.0
===================================
tab: vertimus-1-0 (2008-03-21)

* XHTML and CSS pass the W3C validator
* New tool to download all PO files of a release
* Show informations about other modules with the same name in the
module page (Javascript)
* Move old files in a backup directory
* Add some maintenance scripts
* Display the authorized extensions in module page
* Add a search tool
* Reduce require_once calls
* Add informations about APC installation
* FIX Backup of files
* FIX No error message when the extension file isn’t valid
* FIX Web site title translation
* FIX call to date_default_timezone_set
* Fix #189903 - RSS feed error

New and last release wrote in PHP! Take it while it’s hot! The next version will be in Python to integrate features from Damned-lies and Transifex.



Waf build system

by Stéphane

17 03 2008

I integrate the Waf build system to libmodbus. Waf is a really nice project I hope Thomas Nagy will have time to update his documentation. The code is small and really easy to browse but the most impressive difference is the execution time:

time (./autogen.sh && ./configure && make && sudo make install)
=> 21.576s (with AM_DISABLE_STATIC)

time (./waf configure && ./waf build && sudo ./waf install)
=> 2.779s

The output is nice (compact and colorful), you can also use Waf in daemon mode (works with inotify) and each time a file of your project is modified, Waf build the new component. The size of the configure script is ~700Kb and if you choose to embed Waf it’s only 100Kb. Last but not least, your build system doesn’t check for Fortran anymore :).

Thanks to Ali Sabil for his demonstration of Waf at FOSDEM!



Performance of a PHP application with APC

by Stéphane

9 03 2008

To evaluate the performance of Vertimus with a opcode cache like APC, I used xdebug and Kcachegrind.
The results are really interesting, without APC, the index page has a total time cost of 191 032, the Zend Framework requires to use many classes, PHP is not really fast to parse and execute this code :

Vertimus without APC

and with APC, the total time cost is only 123 904:
Vertimus with APC

The CPU load is reduced by 36% but you need a bit of memory to store cache data (30 Mo by default). The results has been obtained with APC 3.0.16 and the following configuration:

;;;;;;;;;;;;;;;;;;
; APC ;
;;;;;;;;;;;;;;;;;;
extension=apc.so
apc.enabled=1
apc.shm_segments=1
apc.shm_size=30
apc.ttl=7200
apc.user_ttl=7200



Cairo-Clock is now really faster!

by Stéphane

3 03 2008

capture-cairo-clock-par-macslow.pngCairo-Clock is CPU intensive and consume around 13 % of CPU on my Core 2 Duo (on one CPU), the workload is spread on Xorg (10 %), compiz.real (2 %) and cairo-clock (1 %) with a smoothing value defined at 30 (default). The second hand drawing is disabled (default).

I wrote some patches to reduce the CPU consumption and now, Cairo-Clock doesn’t appear in ‘top’ list, woooh! The solution was simple, it useless to refresh the clock at 30 Hz if the second hand is not drawn. I also use common code to manage timeout and g_timeadd_seconds reduce the number of interrupts when many applications use this function.

I prefer bzr to git but I published my patches on github so MacSlow can take my patches here: http://github.com/stephane/cairo-clock/tree/master

PS: I couldn’t resist to translate in French (and fix Cairo-Clock for translation).



Vertimus 0.4, ‘Hello, Zend!’

by Stéphane

28 02 2008

Vertimus is an Open Source web tool to manage a workflow of translations, each translation has a status which changes at each step. You can create an account, book any translation and communicate with your team. If you want to catch a glimpse of Vertimus in action, go to http://gnomefr.traduc.org/suivi (used by the French team).

This is the first release with support for localization, two languages are included in this release English (thanks to Phil Frary) and… French of course :).

Now, I think Vertimus is ready for world domination, so I urge you to translate Vertimus into your language and deploy it on a web server near to you (see the INSTALL file to follow the easy steps).


===================================
Overview of changes in Vertimus 0.4
===================================
tag: vertimus-0-4 (2008-01-15)
* based on the Zend Framework:
- Zend_Mail
- Zend_Translate
- Zend_Locale
- Zend_Db (PDO_MySQL)
- Zend_Auth
* new look & feel (CSS)
* separate specific settings in localconfig.php (URL, menu entries)
* translated in English and French
* mail encoded in UTF-8
* show a information message (editable by the administrator)
* use a sort order for the releases
* simpler session management
* the user can reset his password via an activation key
* licence is now GPL only
* cleanups
* FIX #158764 - Mail sent about status being changed to "Translated"
when cancel ing an action
* FIX #174698 - Angle brackets should be converted in comments

To checkout the latest version: bzr co lp:vertimus



HIG Frames with GTK+

by Stéphane

3 02 2008

It’s painful to follow the GNOME Human Interface Guidelines in plain GTK+ so I wrote some functions to make this task easier. I wrote this code one year ago and I’m suprised we don’t have such API in the current GTK+ version.

Screenshot HIG GTK+ Frames

The Launchpad branch is https://code.launchpad.net/gtk-frame-hig. libsexy is maybe the right container for that code.

PS: libmodbus 1.2.3 is out!