Autocomplete Ajax search with Dojo and Zend Framework

by regilero

26 10 2008
autocomplete-ajax-search-with-dojo-and-zend-framework

search filter 2With the new Zend Framework 1.6 we’ve these nice Dojo widgets.

New things lacks documentations most of times. So if you want to build something really usefull like theses nice autocomplete search combobox this example could save you a lot of time.
We assume you have dojo already installed and activated on your views, and that acl verifications are done elsewhere, on your Controller plugins for example.
search filter

First let’s see HTML code (in your view):
<script type="text/javascript">
    dojo.require("dojo.parser");
    dojo.require("dojox.data.QueryReadStore");
    dojo.require("dijit.form.ComboBox");
    dojo.require("dijit.form.FilteringSelect");
    dojo.require("custom.FindAutoCompleteReadStore");
    dojo.require("dijit.form.Form");
    dojo.require("dijit.form.Button");
</script>
<form id="Find_Form" action="/module/foo/edit" method="get" dojoType="dijit.form.Form">
<div dojoType="custom.FindAutoCompleteReadStore" jsId="NameStore" url="/module/foo/find/format/json" requestMethod="get"></div>
<label for="id" class="optional">Recherchez un nom:</label>
<span class="formelement"><select name="id" id="FindByName" hasDownArrow="" store="NameStore" size="25" tabindex="99" autocomplete="1" dojoType="dijit.form.FilteringSelect" pageSize="10" ></select></span>
<span class="actionbuttons"><input id="Find_go" name="Find_go" value="Go:" type="submit" label="go:"dojoType="dijit.form.Button" /></span>
</form>

As we can see you’ll need an additional custom js:
custom.FindAutoCompleteReadStore
This is a really simple js to write, create your custom directory in the same level as dojo or dijit directory and create FindAutoCompleteReadStore.js like that:
dojo.provide("custom.FindAutoCompleteReadStore");
dojo.require("dojox.data.QueryReadStore");
dojo.declare("custom.FindAutoCompleteReadStore", dojox.data.QueryReadStore, {
    fetch:function (request) {
        request.serverQuery = { Find:request.query.name };
        // cal superclass fecth
        return this.inherited("fetch", arguments);
    }
});

Now you’ll need to serve the requested Ajax query (requested by the Dojo store linked with our FilteringSelect or Combobox) : /module/foo/find/format/json
This is the method ‘findAction’ in the Controller ‘foo’ on module ‘module’.
But first let’s see the preDispatch function of this controller where we handle the format/json instruction to switch in Ajax mode:

public function preDispatch()
{
    $contextSwitch = $this->_helper->getHelper('contextSwitch');
    $contextSwitch->setAutoJsonSerialization( true );
    $contextSwitch->addActionContext('find', 'json');
    $contextSwitch->initContext();
}

So now let’s write the find function:

public function findAction()
{
    // handle filtering of recieved data
    $replacer = new Zend_Filter_pregReplace('/\*/','%');
    // emulate alpha+num filter with some more characters enabled
    //**** http://www.regular-expressions.info/unicode.html ****
    // \p{N} --> numeric chars of any language
    // \s -> withespace
    //\x0027 : APOSTROPHE

    //\x002C : COMMA
    //\x0025% : % in UTF-8 and not in utf-8
    //\x002D : HYPHEN / MINUS
    //\x005F : UNDERSCORE
    //\. DOT
    $mylimit = new Zend_Filter_pregReplace('/[^\p{L}\p{N}\s\x0027\x002C\x002D\x005F\x0025%\.]/u’,”);
    $filters = array(
        ’*’     => ‘StringTrim’
        ,’Find’ => array(
            ’StripNewlines’
            ,$replacer
            ,$mylimit
            ,’StripTags’
        )
        ,’start’ => ‘Int’
        ,’count’ => ‘Int’
    );
    $validators =array();
    $input = new Zend_Filter_Input($filters, $validators, $_GET);
    $find = $input->getUnescaped(’Find’);
    if (empty($find)) $find = ‘%’;
    $start = intval($input->getUnescaped(’start’));
    if (empty($start)) $start = 0;
    $count = intval($input->getUnescaped(’count’));
    if (empty($count)) $count = 3;
    // get the model, here you should adjust with the way you work
    // then make your query with limits
    $this->_modeltable = new My_Zend_Db_Table_Foo($this->db)
    $fieldid = ‘my_id_field’;
    $fieldident = ‘my_name_field’;
    $select = $this->_modeltable->select();
    $db = $this->_modeltable->getAdapter();
    $select->where($db->quoteinto($db->quoteIdentifier($fieldident).’ LIKE ?’, $find));
    $select->limit($count, $start);
    $rows= $this->_modeltable->fetchAll($select);
    $rowsarray = $rows->ToArray();
    $finalarray=array();
    foreach ($rowsarray as $row)
    {
        $key = $row[$fieldid];
        $finalarray[$key] = $row[$fieldident];
    }
//Zend_Debug::dump($finalarray);
//die(__METHOD__);
    $this->_helper->autoCompleteDojo($finalarray);
}

And it should be sufficient, pffiuu.
But… there’s one remaining problem after that. We put the search autocomplete inside a form and we wanted the ‘go’ button to send a request to something like that:

/module/foo/edit/id/1245 OR /module/foo/edit?id=1245

But we’ll have something like:

/module/foo/edit?id=THE NAME

too bad…

To get it done I had to change one thing in Zend Framework library on the Zend/Controller/Action/Helper/AutoCompleteDojo.php Helper:

62 public function prepareAutoCompletion($data, $keepLayouts = false)
63 {
64 $items = array();
65 foreach ($data as $key => $value) {
66 $items[] = array(’label’ => $value, ‘name’ => $value, ‘key’ => $key);
67 }
68 $final = array(
69 ‘identifier’ => ‘key’,
70 ‘items’ => $items,
71 );
72 return $this->encodeJson($final, $keepLayouts);
73 }

Line 66 ‘key’ is added on the item and line 69 ‘identifier’ is set to ‘key’ and not ‘name’. ‘identifier’ is used by the Dojo Filtering Select to decide which field will be used for the form, for more info see dojo book page and search ‘abbreviation’. There’s also a bug talking about that for Zend Framework, to get other solutions or info on the way it will be fixed later look here



[OFF] Et si j’étais un prof de philo ?

by pounard

19 10 2008

.. je pense que je donnerais comme sujet de dissertation des citations chinoises !

Et oui, en balladant sur le net, j'en ai repéré une ou deux que je trouve tout à fait juste, mais libre à chacun d'avoir son opinion sur le sujet.

Petite note, elles sont toutes de Cofucius (étonnant tiens!).

en lire plus


Zend Pdf and PNG transparency issue

by alf

5 06 2008

Currently working on PDF generation, I’ve noticed that drawing a big PNG image with transparency (about 600×800, 150kB) brings the script computation up to 20 seconds. I’ve raised the bug in the Zend Framework issue tracker and for now, don’t forget to remove transparency in your background image if you can.

See the bug at :  http://framework.zend.com/issues/browse/ZF-3392



Wrapping text for Zend Pdf

by alf

4 06 2008

A common issue in Zend_Pdf is to wrap text in a box. I’ve found partial solutions such as wrapping text each 80 characters for instance but the line width can vary regarding the font and the character width. Since we can’t rely on the character count unless using a monospaced font, we have to wrap text on the real box width.

In partial solutions, I’ve found a function which computes the real width of a string according to the font and the font size. By aggregating every chunks, I’ve made my getWrappedText() method which returns a string with the correct \n :

protected function getWrappedText($string, Zend_Pdf_Style $style,$max_width)
{
    $wrappedText = '' ;
    $lines = explode("\n",$string) ;
    foreach($lines as $line) {
         $words = explode(' ',$line) ;
         $word_count = count($words) ;
         $i = 0 ;
         $wrappedLine = '' ;
         while($i < $word_count)
         {
             /* if adding a new word isn't wider than $max_width,
             we add the word */
             if($this->widthForStringUsingFontSize($wrappedLine.' '.$words[$i]
                 ,$style->getFont()
                 , $style->getFontSize()) < $max_width) {
                 if(!empty($wrappedLine)) {
                     $wrappedLine .= ' ' ;
                 }
                 $wrappedLine .= $words[$i] ;
             } else {
                 $wrappedText .= $wrappedLine."\n" ;
                 $wrappedLine = $words[$i] ;
             }
             $i++ ;
         }
         $wrappedText .= $wrappedLine."\n" ;
     }
     return $wrappedText ;
}
/**
 * found here, not sure of the author :
 * http://devzone.zend.com/article/2525-Zend_Pdf-tutorial#comments-2535
 */
 protected function widthForStringUsingFontSize($string, $font, $fontSize)
 {
     $drawingString = iconv('UTF-8', 'UTF-16BE//IGNORE', $string);
     $characters = array();
     for ($i = 0; $i < strlen($drawingString); $i++) {
         $characters[] = (ord($drawingString[$i++]) << 8 ) | ord($drawingString[$i]);
     }
     $glyphs = $font->glyphNumbersForCharacters($characters);
     $widths = $font->widthsForGlyphs($glyphs);
     $stringWidth = (array_sum($widths) / $font->getUnitsPerEm()) * $fontSize;
     return $stringWidth;
 }

then you can draw the text easily :

$y = 700;
$lines = explode("\n",$this->getWrappedText($text,$style_text,400)) ;
foreach($lines as $line)
{
    $page2->drawText($line, 140, $y);
    $y-=15;
}


Integration of Scriptaculous slider in Zend framework

by alf

29 05 2008

For a customer need, I’ve integrated the scriptaculous slider as shown here in Zend framework.

To achieve this, I’ve created a formSlider view helper which is then used by a slider form element.

The view helper and form element code is here. To allow ZF to find the helper, don’t forget to add path to helper in your bootstrap :

$layout->getView()->addHelperPath('Mc/View/Helper','Mc_View_Helper') ;

You need of course to include scriptaculous and prototype to make it work (available here and here) :

<script src="/prototype-1.6.0.2.js" type="text/javascript"></script>
<script src="/scriptaculous.js" type="text/javascript"></script>

Then, you can create a slider like this :

require_once('Mc/Form/Element/Slider.php') ;
$my_form->addElement(new Mc_Form_Element_Slider('my_element')
    ,'my_element') ;
$my_form->my_element->setLabel('My element : ')
    ->setMin(10)
    ->setMax(30)
    ->setStep(2);

and retrieve the value as usual :

$my_form->my_element->getValue() ;

And if you want to override the default style in your css, don’t forget the !important argument :

div.slider {
    width:256px !important;
    margin:10px 0 !important;
    background-color:lightgray !important;
    height:15px !important;
    position: relative !important;
 }

div.slider div.handle {
     width:10px !important;
     height:15px !important;
     background-color:darkgray !important;
     cursor:move !important;
     position: absolute !important;
 }


Secure authentication without SSL (part2)

by alf

21 02 2008

After chatting with my colleagues, we noticed that my CRAM can be worth than nothing.

First, I have forgotten to precise what was the goal. It doesn’t prevent from stealing session by a man in the middle. The goal was to avoid seeing password on the network (if you use the same for all websites) and to prevent replay attack. The CRAM prevent replay by asking for a response with a different salt each time but it implies that the client and the server share the password. I was mistaken with this approach.

It’s a known and good practice to store a password hash instead of plain text password since a read access to the database doesn’t allow an attacker to authenticate since he sends the password in plain text and the server makes the hash. With my CRAM, even if you only have the hash, you can authenticate. That’s the security hole. Since you can’t prevent from a man in the middle attack without SSL, the CRAM seems useless.

But we still want to prevent replay and showing password. First, let’s come back to a basic authentication : we send username and password in plain text, the server makes the hash of the password and compare it with the one in database. To prevent the plain text exchange, we will store a double hash of the password in database : sha1(sha1(password)). The client sends sha1(password) and the server apply a new sha1 before comparing. Concerning the replay, we will use the CRAM. The server sends a unique salt for each try and the client still sends sha1(password) for authentication. The client will also send sha1(salt+md5(password)) as the challenge response. Then the server can authenticate the client with sha1(password) and ensure uniqueness of the challenge with the challenge-response.This implies to store sha1(sha1(password)) and md5(password) in database.

We have the best of the two approach. A man in the middle won’t see the password in plain text and won’t be able to replay the authentication because of the CRAM. He still is able to steal the session, but as long as it is valid. If the valid user logs out, the session becomes invalid. If an attacker has access to the database, he will find md5(password) and sha1(sha1(password)) what will allow him to answer to the CRAM but not to the basic authentication (which wait for sha1(password)).

This method remains low secure but I can’t see anything better without SSL. Like Bram sugessted, I should add a basic plain text authentication by default if javascript is disabled.



Secure authentication without SSL

by alf

18 02 2008

Last days, I’ve implemented authentication in Wisss. The goal is to have a secure authentication without the need to use SSL. I’ve then choose a basic Challenge Response Authentication Mechanism which prevent to send password in clear text.

The mechanisme is simple :

  1. The server send a unique salt, which can be used only once
  2. The client answer with its username and SHA1(salt+SHA1(password))
  3. The server retrieves user’s password SHA1 from database and do SHA1(salt,sha1_password)
  4. if the two match, the user is authenticated

Here is the code for the server (it’s inside a Zend controller, but it can easily be implemented for all framework/language) :

The CRAM authentication adapter for Zend_Auth :

class Wisss_Auth_Adapter_BasicCram implements Zend_Auth_Adapter_Interface
    protected $login ;
    protected $digest ;

    public funtion __construct($login,$digest)
    {
        $this->login = $login ;
        $this->digest = $digest ;
    }

    public function authenticate()
    {
        $auth_session = new Zend_Session_Namespace('auth');
        // retrieve salt from session and erase it since it's used once
        $salt = $auth_session->salt ;
        unset($auth_session->salt) ;
        // digest received from client
        $cram_from_client = $this->digest ;
        // retrieve password sha1 digest from database for the user provided
        require_once('Wisss/Dao/Factory/Abstract.php') ;
        $dao_user = Wisss_Dao_Factory_Abstract::getDao('User','core') ;
        $user = $dao_user->findBy(array('login' => $this->login)) ;
        $sha1_password = $user[0]->getPassword() ;
        // compute the CRAM digest
        $cram_from_server = sha1($salt.$sha1_password) ;
        if($cram_from_client == $cram_from_server) {
            return new Zend_Auth_Result(Zend_Auth_Result::SUCCESS,$this->login) ;
        } else {
            return new Zend_Auth_Result(Zend_Auth_Result::FAILURE,$this->login) ;
        }
    }
 }

The code of the controller :

class LoginController extends Zend_Controller_Action
{
    public function indexAction()
    {
        $salt = md5(uniqid(rand(), true)) ;
        $auth_session = new Zend_Session_Namespace('auth');
        $this->view->assign('login_salt',$salt) ;
        $auth_session->salt = $salt ;
        $auth_session->referer = $_SERVER['HTTP_REFERER'] ;
    }

    public function authenticateAction()
    {
        $auth_session = new Zend_Session_Namespace('auth');
        require_once 'Zend/Auth.php';
        $auth = Zend_Auth::getInstance();
        // Set up the authentication adapter
        require_once('Wisss/Auth/Adapter/BasicCram.php') ;
        $authAdapter = new Wisss_Auth_Adapter_BasicCram($this->_getParam('login'), $this->_getParam('password'));
        // Attempt authentication, saving the result
        $result = $auth->authenticate($authAdapter);
        if (!$result->isValid()) {
            // Authentication failed
            $this->_redirect('/login') ;
        } else {
            // Authentification succeeded
            // regenerate id to prevent session fixation after successfull authentication
            Zend_Session::regenerateId();
            $referer = $auth_session->referer ;
            unset($auth_session->referer) ;
            if($referer != $_SERVER['HTTP_REFERER']) {
                $this->_redirect($referer) ;
            }
        }
    }
}

and finally, the client code (it requires the Prototype Javascript framework) :

<div>
<form action="login/authenticate" method="POST" onSubmit="cram()">
<?php echo $this->formHidden('login_salt',$this->login_salt) ?>
<fieldset>
<div><label for="login">Login : </label><?php echo $this->formText('login') ?></div>
<div><label for="password">Password : </label><?php echo $this->formPassword('password') ?></div>
<div><input type="submit" name="authenticate" value="login"/></div>
</fieldset>
</form>
</div>

And the associated js :

function cram()
{
    $('password').value = sha1Hash($('login_salt').value+sha1Hash($('password').value)) ;
    $('login_salt').value = "" ;
}

// © 2002-2005 Chris Veness
// http://www.movable-type.co.uk/scripts/sha1.html
function sha1Hash(msg)
{
    // constants [§4.2.1]
    var K = [0x5a827999, 0x6ed9eba1, 0x8f1bbcdc, 0xca62c1d6];

    // PREPROCESSING
    msg += String.fromCharCode(0x80); // add trailing '1' bit to string [§5.1.1]

    // convert string msg into 512-bit/16-integer blocks arrays of ints [§5.2.1]
    var l = Math.ceil(msg.length/4) + 2;  // long enough to contain msg plus 2-word length
    var N = Math.ceil(l/16);              // in N 16-int blocks
    var M = new Array(N);
    for (var i=0; i<N; i++) {
        M[i] = new Array(16);
        for (var j=0; j<16; j++) {  // encode 4 chars per integer, big-endian encoding
            M[i][j] = (msg.charCodeAt(i*64+j*4)<<24) | (msg.charCodeAt(i*64+j*4+1)<<16) |
            (msg.charCodeAt(i*64+j*4+2)<<8) | (msg.charCodeAt(i*64+j*4+3));
        }
    }
    // add length (in bits) into final pair of 32-bit integers (big-endian) [5.1.1]
    // note: most significant word would be ((len-1)*8 >>> 32, but since JS converts
    // bitwise-op args to 32 bits, we need to simulate this by arithmetic operators
    M[N-1][14] = ((msg.length-1)*8) / Math.pow(2, 32); M[N-1][14] = Math.floor(M[N-1][14])
    M[N-1][15] = ((msg.length-1)*8) & 0xffffffff;

    // set initial hash value [§5.3.1]
    var H0 = 0x67452301;
    var H1 = 0xefcdab89;
    var H2 = 0x98badcfe;
    var H3 = 0x10325476;
    var H4 = 0xc3d2e1f0;

    // HASH COMPUTATION [§6.1.2]

    var W = new Array(80); var a, b, c, d, e;
    for (var i=0; i<N; i++) {
        // 1 - prepare message schedule 'W'
        for (var t=0;  t<16; t++) W[t] = M[i][t];
        for (var t=16; t<80; t++) W[t] = ROTL(W[t-3] ^ W[t-8] ^ W[t-14] ^ W[t-16], 1);
        // 2 - initialise five working variables a, b, c, d, e with previous hash value
        a = H0; b = H1; c = H2; d = H3; e = H4;

        // 3 - main loop
        for (var t=0; t<80; t++) {
            var s = Math.floor(t/20); // seq for blocks of 'f' functions and 'K' constants
            var T = (ROTL(a,5) + f(s,b,c,d) + e + K[s] + W[t]) & 0xffffffff;
            e = d;
            d = c;
            c = ROTL(b, 30);
            b = a;
            a = T;
        }

        // 4 - compute the new intermediate hash value
        H0 = (H0+a) & 0xffffffff;  // note 'addition modulo 2^32'
        H1 = (H1+b) & 0xffffffff;
        H2 = (H2+c) & 0xffffffff;
        H3 = (H3+d) & 0xffffffff;
        H4 = (H4+e) & 0xffffffff;
    }

    return H0.toHexStr() + H1.toHexStr() + H2.toHexStr() + H3.toHexStr() + H4.toHexStr();
}

//
// function 'f' [§4.1.1]
//
function f(s, x, y, z)
{
    switch (s) {
    case 0: return (x & y) ^ (~x & z);           // Ch()
    case 1: return x ^ y ^ z;                    // Parity()
    case 2: return (x & y) ^ (x & z) ^ (y & z);  // Maj()
    case 3: return x ^ y ^ z;                    // Parity()
}
}

//
// rotate left (circular left shift) value x by n positions [§3.2.5]
//
function ROTL(x, n)
{
    return (x<<n) | (x>>>(32-n));
}

//
// extend Number class with a tailored hex-string method
//   (note toString(16) is implementation-dependant, and
//   in IE returns signed numbers when used on full words)
//
Number.prototype.toHexStr = function()
{
    var s="", v;
    for (var i=7; i>=0; i--) { v = (this>>>(i*4)) & 0xf; s += v.toString(16); }
    return s;
}