TUTORIAL: Efficient Storage of Option Choices Facilitates Searching & Retrieval - Page 2

molotov · #11 (**permalink**) 07-29-2007, 02:19 PM

Quote:

Originally Posted by TeraTask

See Want to Be a Computer Scientist? Forget Maths for the motivation behind this solution. There is a way of doing this better than just assigning the numbers above. I just don't have time to write it up.

Ah! That is very clever. I was trying to work out a way using primes or something with the idea that if the category % 2 = 0, you increase cost figuring out which categories or doing another lookup, but if my assumption that most books are only in one category is true, it might be worth the extra cost. I really don't know anything about the dataset though, so it was a shot in the dark.

Also, when working with powers of 2, shifting bits is more than twice as fast as pow().

TeraTask · #12 (**permalink**) 07-29-2007, 04:47 PM

Quote:

Originally Posted by molotov

Also, when working with powers of 2, shifting bits is more than twice as fast as pow().

Superior point!!!! Thanks.

molotov · #13 (**permalink**) 08-24-2007, 11:52 AM

I found something interesting. It looks like the problem we're having is limited to the array keys, not PHP or MySQL. We're using unsigned BIGINTs, so that's not a problem until we get to 2^54 apparently, and if you run this snippet below, you'll see PHP doesn't have a problem dealing with large numbers:

PHP Code:

  for ($i=0;$i<128;$i++)
{
    echo "$i = " . pow(2, $i) . "\n";
}

BUT, when you bit-shift, you run into the previous problem:

PHP Code:

  for ($i=0;$i<128;$i++)
{
    echo "$i = " . (2 << $i) . "\n";
}

I suppose that can be explained by shifting out of the memory blocks where these numbers are residing. Neat.

So anyway, what if we did something like this?

PHP Code:

    $my_subjects[] = array('name'    => 'subject',
                       'options' => array(''.pow(2,0).''  => 'Arts & Photography',        ''.pow(2,1).''  => 'Audiocassettes',            ''.pow(2,2).''  => 'Audio CDs', 
                                          ''.pow(2,3).''  => 'Audio Downloads',           ''.pow(2,4).''  => 'Bargain Books',             ''.pow(2,5).''  => 'Biographies & Memoirs', 
                                          ''.pow(2,6).''  => 'Business & Investing ',     ''.pow(2,7).''  => 'Children\'s Books',         ''.pow(2,8).''  => 'Christian Books', 
                                          ''.pow(2,9).''  => 'Comics & Graphic Novels',   ''.pow(2,10).'' => 'Computers & Internet',      ''.pow(2,11).'' => 'Cooking, Food & Wine', 
                                          ''.pow(2,12).'' => 'e-Books',                   ''.pow(2,13).'' => 'Entertainment',             ''.pow(2,14).'' => 'Español', 
                                          ''.pow(2,15).'' => 'Gay & Lesbian',             ''.pow(2,16).'' => 'Health, Mind & Body',       ''.pow(2,17).'' => 'History', 
                                          ''.pow(2,18).'' => 'History',                   ''.pow(2,19).'' => 'Horror',                    ''.pow(2,20).'' => 'Literature & Fiction', 
                                          ''.pow(2,21).'' => 'Mystery & Thrillers',       ''.pow(2,22).'' => 'Nonfiction',                ''.pow(2,23).'' => 'Outdoors & Nature ', 
                                          ''.pow(2,24).'' => 'Parenting & Families',      ''.pow(2,25).'' => 'Professional & Technical',  ''.pow(2,26).'' => 'Reference', 
                                          ''.pow(2,27).'' => 'Religion & Spirituality',   ''.pow(2,28).'' => 'Romance',                   ''.pow(2,29).'' => 'Science', 
                                          ''.pow(2,30).'' => 'Science Fiction & Fantasy', ''.pow(2,31).'' => 'Sports',                    ''.pow(2,32).'' => 'Teen', 
                                          ''.pow(2,33).'' => 'Travel',                    ''.pow(2,34).'' => 'Women\'s Fiction'
                                         ));
                                         
                                         
                                         var_dump($my_subjects);

It's kind of a hacky way of turning those numbers into strings. If anyone knows a better way, I'd love to hear it just because the type-abuse above feels so wrong

So now we have array indexes that are number strings, but can be shoved into MySQL as BIGINTs. Thoughts?

Here's the whole thing for easy copy-paste:

PHP Code:

  <?php

echo "<pre>";

for ($i=0;$i<128;$i++)
{
    echo "$i = " . pow(2, $i) . "\n";
}


for ($i=0;$i<128;$i++)
{
    echo "$i = " . (2 << $i) . "\n";
}


  $my_subjects[] = array('name'    => 'subject',
                       'options' => array(''.pow(2,0).''  => 'Arts & Photography',        ''.pow(2,1).''  => 'Audiocassettes',            ''.pow(2,2).''  => 'Audio CDs', 
                                          ''.pow(2,3).''  => 'Audio Downloads',           ''.pow(2,4).''  => 'Bargain Books',             ''.pow(2,5).''  => 'Biographies & Memoirs', 
                                          ''.pow(2,6).''  => 'Business & Investing ',     ''.pow(2,7).''  => 'Children\'s Books',         ''.pow(2,8).''  => 'Christian Books', 
                                          ''.pow(2,9).''  => 'Comics & Graphic Novels',   ''.pow(2,10).'' => 'Computers & Internet',      ''.pow(2,11).'' => 'Cooking, Food & Wine', 
                                          ''.pow(2,12).'' => 'e-Books',                   ''.pow(2,13).'' => 'Entertainment',             ''.pow(2,14).'' => 'Español', 
                                          ''.pow(2,15).'' => 'Gay & Lesbian',             ''.pow(2,16).'' => 'Health, Mind & Body',       ''.pow(2,17).'' => 'History', 
                                          ''.pow(2,18).'' => 'History',                   ''.pow(2,19).'' => 'Horror',                    ''.pow(2,20).'' => 'Literature & Fiction', 
                                          ''.pow(2,21).'' => 'Mystery & Thrillers',       ''.pow(2,22).'' => 'Nonfiction',                ''.pow(2,23).'' => 'Outdoors & Nature ', 
                                          ''.pow(2,24).'' => 'Parenting & Families',      ''.pow(2,25).'' => 'Professional & Technical',  ''.pow(2,26).'' => 'Reference', 
                                          ''.pow(2,27).'' => 'Religion & Spirituality',   ''.pow(2,28).'' => 'Romance',                   ''.pow(2,29).'' => 'Science', 
                                          ''.pow(2,30).'' => 'Science Fiction & Fantasy', ''.pow(2,31).'' => 'Sports',                    ''.pow(2,32).'' => 'Teen', 
                                          ''.pow(2,33).'' => 'Travel',                    ''.pow(2,34).'' => 'Women\'s Fiction'
                                         ));
                                         
                                         
                                         var_dump($my_subjects);

?>

EDIT: found this: PHP: bcpow - Manual

bcpow() would get around having to do that '' dance.

molotov · #14 (**permalink**) 09-19-2007, 02:06 PM

Bump.

Still curious to the solution for this issue.

TeraTask · #15 (**permalink**) 09-19-2007, 02:09 PM

I just may get back to this shortly. I found PHP: GMP Functions - Manual and want to write this up in a generic class, but as indicated elsewhere in these forums, I have an enormous amount of work on my hands right now.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

The Following User Says Thank You to molotov For This Useful Post:
TeraTask (07-29-2007)