This is the 303 code specification in the HTTP/1.1 RFC:

The response to the request can be found under a different URI and SHOULD be retrieved
using a GET method on that resource. This method exists primarily to allow the output of a
POST-activated script to redirect the user agent to a selected resource. The new URI is not
a substitute reference for the originally requested resource. The 303 response MUST NOT be
cached, but the response to the second (redirected) request might be cacheable.

OK, they are implying that you request “something”, but instead of that “something”, you are being redirection to “something else”. The 303 response should not be cached, but the location the 303 points to, can be cached. This makes sense and still is valid in redirecting a user to the actual resource after the job in the queue has finished. They even provide a use-case, where a POST to for instance a form, will redirect the user to another page. Key here, I think at least, is the fact that the new URI is not a substitute for the old URI. In other words: it is not the same resource, but on another place. The URI points to a completely different resource.

The next sentences:

The different URI SHOULD be given by the Location field in the response. Unless the
request method was HEAD, the entity of the response SHOULD contain a short hypertext
note with a hyperlink to the new URI(s).

So we should return the actual resource through the “Location:” field, and we should return a bit of hypertext with a href or rel to the new uri (there can be multiple!).

I still think it makes sense to use this code since we are doing two things after completing our asynchronous operation:

1. We want to notify the user that the operation has been completed.
2. We want to redirect the user to the OUTPUT of the operation (ie, the newly created resource).

Both can be fulfilled with a 303 response.

Now, the suggestion came that you should 301 for this behavior instead of 303. So let’s read the docs:

The requested resource has been assigned a new permanent URI and any future references
to this resource SHOULD use one of the returned URIs. Clients with link editing
capabilities ought to automatically re-link references to the Request-URI to one or
more of the new references returned by the server, where possible. This response is
cacheable unless indicated otherwise.

It states that a 301 is a permanent redirection of the resource to another location (Like a “we’re moved!” sign). However, the actual resource we are asking when doing a /queue/12345 is not the resource it created, but is the actual queue-task itself. And that queue-task hasn’t moved! When we do a GET /queue/12345 and return a 301 Moved -> /my/new/resource, it implies that this data was originally found on /queue/12345, but this was never the case! The rest of the sentences aren’t really important in deciding between a 301 or a 303.

The new permanent URI SHOULD be given by the Location field in the response. Unless
the request method was HEAD, the entity of the response SHOULD contain a short
hypertext note with a hyperlink to the new URI(s).

Again, this is the same as with a 303, so nothing strange here.

If the 301 status code is received in response to a request other than GET or HEAD,
the user agent MUST NOT automatically redirect the request unless it can be confirmed
by the user, since this might change the conditions under which the request was issued.

This is probably here to make sure that we don’t do a non-safe action to a resource which might have moved in the meantime. But I think this should be handled with etags and if-(not-)modified headers anyway. By the way, there is a draft of a status code that makes it mandatory to use conditional headers when accessing or modifying resources (428 Precondition Required).

So in conclusion: I’m still not convinced, at least, according to how I interpret the http spec, that we should use 301 here, instead of 303. If you see it differently (and I know some of you will :p), I’d like to find out why a 301 is better in your view. But maybe the most important thing about this discussion, and thus this blog-post, is that interpretation of the rules can be difficult on occasion, and maybe the rules are ambiguous enough that there are no clear winners on some of these discussions. However, it’s good to discuss these things, it makes the whole REST & HTTP way thinking stronger in the end, which in a API-centric world we are entering is definitely not a bad thing :)

Whiskyweb – Edinburgh – Scotland

Whiskyweb is a truly amazing conference. Especially if you know that this conference has been setup from scratch in about 3 months! Even most of the professional conferences out there cannot even organize what Juokaz, Micheal, Max, Paul and all the others involved have done in such a short time. Whiskyweb was held in Edinburgh, Scotland, which is one of the nicest cities I’ve recently been too. The city center consists of many old buildings and with all the pubs on the Royal Mile, you can’t go wrong.

On the first day I arrived, I was just in time for the speakers diner where afterwards we all did a sort-of pub-crawl along many of the pubs on the Royal Mile. The next day we had a 15 minute walk to the venue together with BinaryKitten and Geekie who where staying in the same hotel as well. The conference venue was truly awesome: a big church modified for venues with room for a lot of attendees. Unfortunately, the registration process wasn’t very quick, so the introduction was already started by the time we got our passes. On the conference itself, there were many speakers from all over the world speaking about lots of different subjects. My own talk was in the second room (I get that a lot :p), which was warm and dark, and together with some fairly complex theoretical stuff, it doesn’t make a really nice combo. However, most people enjoyed it after a long day of the conference though. And off course, what’s a conference without a flying radio-controlled shark? Nothing off course, which was the reason whiskyweb was guarded by a flying shark (minus the laserbeam on its head). The day ended with a whisky-workshop where we could taste all kind of different wishky’s, and got an explanation on whisky in general.

The second day was all about hackathons. It was more a do-what-you-like venue, but groups were form quickly en0ugh doing all kinds of stuff. Josh Holmes gave a workshop on speaking, I did some introduction on Puppet and Lorna hacked away on the joind.in API so others could actually use them for their own projects they were working on. The coolest projects I’ve seen that day was from two guys who wanted to do some work with Puppet and Vagrant, and managed to setup the joind.in application through vagrant, which is now in the process of being added to the master branch (yay!). And this actually shows you what a hackathon is all about.

4developers – Poznan – Poland

The following day I had to fly over to Poland for the 4developers conferences. This was my second time around at 4developers although the last time was in Warsaw. A lot of the speakers from Whiskyweb did an appearance at 4developers as well, so it was great fun to see the same group of people again, just in another country (that happens a lot by the way).

The 4developers also started with a really nice speakers dinner on the market square in the center of Poznan (at which we were drinking beer the whole afternoon already). Off course, a good speakers dinner in Poland cannot be ended without having a few bottles of wodka. There was even a point that we though that some of the speakers would never make it on time for their talk (or at least, not sober :p).

Fortunately, the venue was just a 5 minute walk away from our hotel and it was really nice venue. This time, the PHP track was the biggest track, with some small java, scaling and management tracks on another floor. The venue looks really great and futuristic and there 4developers bug mascot looked really nice (we’ve got the t-shirt). Maybe not every track was properly programmed though: Rowan Merewood’s track was pretty popular but was in the IT -track, meaning there wasn’t even enough room for all the attendees.  At this conferences, both Skoop and I did our first joint talk about “the why and how of learning” in the PHP track, which was received really good by the audience. Off course, we need to do some more fine-tuning, but expect this talk at a conference near you :).  Since I did two talks, my second talk was about one of my favorite subjects: Puppet. Even though it was in a small room, the room was pretty full and most people seemed to enjoy the talk and learned a thing or two about configuration management. After the conference, we all did another speakers-dinner (yay!) which was quite nice as well. Back in the hotel we went over for a beer at the bar.

The next day we had to leave, but our taxi-ride to the airport was not there until 5 so we had time enough to spend our day in the center of Poznan as well. So with a gang of 5 we went into town for a beer and something to eat (not as easy as it might sound) and end up enjoying the “fake jet-set life” on a nice day in the sun in Poznan.

Mail.ru Techforum – Moscow – Russia

With a whopping 2 days time off back home, I had to catch my next flight to Moscow. Never been there, so I really didn’t know what to expect. Actually, to be honest, I was expecting a lot, but not traffic jams. It took the taxi around 2.5 hours for a 40km ride between the airport and Moscow city center (at least I had time to look around).

Luckily, some familiar faces like JP Mens and Kris Buyteart where also present at the conference, so when I checked in the hotel, they were all sitting in the lobby enjoying some beer. After dropping off my luggage in my room (room number 404, and still able to find it!), we went out for a nice Russian dinner. After getting back in the hotel at around 10, Garret Honeycutt and I decided to go for some sightseeing, so we went up to the Kremlin and the red square late in the evening, which was absolutely an awesome sight!

The next morning we were picked up by Kate (Екатерина), who escorted us to the venue. This conference was organized by one of the biggest providers of europe: mail.ru, and it showed: it was a massively impressive conference where in total over 1100 attendants have shown up (mind you: it was a free event as well!). All the talks were done in Russian, and all the talks from the English speakers were translated in real time through an interpreter (very U.N. like). As for the rest, the conference itself was just awesome: photographers taking pictures of everybody and everything, which were printed on the fly so you could get a hard copy of those pictures if you wanted (for free). Bottled water with the logo and date of the conference, personalized badges for everybody with a QR code for easy scanning and exchange and even the talks were broadcasted into the lobby so others could see them as well. Oh, and we could give away a free ipad to the person with the best question for each presentation (which means about 22 ipads in total to give away :)).

Unfortunately, I had to rush to the airport after my talk in order to catch my flight so I didn’t time enough to answers everybodies question after the talk (I’m really sorry guys, if i’m there next year I will make sure I have time enough to answer ALL your questions), so after running 10 minutes late already, plus doing a quick 5 minute interview, I was on route back to the airport after just over 24 hours of Moscow. And yes, traffic was hell again, and it took around 2 hours to get to the airport, just in time for catching the flight with a whopping 20 minutes to spare.

So.. it’s a few weeks off, doing some meetups in between, so don’t miss out on the PHPBenelux meetup on the first of may, and a rerun at may 3rd at Competa. Next up are a social / holiday-esque trip to Verona during the phpday Italy, and off course PHP|tek in Chicago plus the DPC in Amsterdam where I will be doing quite a number of talks on an even broader range of subjects. Don’t miss out!

Defining the problem

So we already know the solution will be a Bloom Filter, but what is the problem they can solve? Suppose you have a book which obviously contains a lot of words. My problem: I want to know if a certain word is actually inside the book. I don’t care where it is, or how many times it’s inside the book: I just want to know if this book contains this specific word or not. As a good PHP developer, you know PHP has got a fair amount of array functionality, so let’s try and use those:

$words = array();
$book = file_get_contents("book.txt");
foreach (split(" ",$book) as $word) {
  $words[strtolower($word)] = 1;
}

if (array_key_exists("supercalifragilisticexpialidocious", $words)) {
  print "Yes, this word is used in the book";
} else{
  print "No, this book is probably not a Mary Poppins book";
}

Introducing the bloom filter

How does it work

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
-----------------------------------------
                    1 1 1 1 1 1 1 1 1 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0

• data: “key1″
• k1(key1) = 4
• k2(key1) = 3
• k3(key1) = 15

0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
-----------------------------------------
                    1 1 1 1 1 1 1 1 1 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0

• data: “another key”
• k1(key1) = 8
• k2(key1) = 19
• k3(key1) = 3

0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0
-----------------------------------------
                    1 1 1 1 1 1 1 1 1 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0

Error rates

Conclusion

Part 5: creating setters

Let’s take a look at the source from this commit: https://github.com/jaytaph/phpshout/blob/99fed246064b5046ec18b848bf779359353268a0/shout.c

Creating the setters is just as easy as creating the getters. The same applies with the setters: we can use a generic function for setting data (we need a few, depending on the type we need to set), but it all work the same way.

Check out line 161 to 212 on the generic setters. Notice that on line 197 we are getting a long integer from PHP, but we need to store a short int. So we mask only the short in part (0xFFFF). This means we can still add a value that is higher than 65535, but it will gets converted to a short. It also would be possible to check the value, and if the value is higher than 65535, it will throw an error.

Like 215 and 216 are special cases, since they will return the error number and error string of the shout library. There are not setters for this. The rest of the functions all have a getters and setters.

Part 6: Other functionality

We’re getting REALLY close in finishing up our extension. All that is left to do is write the functionality for all the non-getter/setter functions. The first one is already there on line 263. This function returns a bool(true) when your  shout object is connected to a stream, or false otherwise. You will see we are doing a mapping on line 271 so it will return either true or false.

Now, let’s take a look at all the functions added. This is done in commit: https://github.com/jaytaph/phpshout/blob/35ac2b32009936e53c3b336efed36a604d37d9f3/shout.c

First off all, I’ve changed the layout a bit for the getters and setters on line 220 to 373. The open(), close() and sync(), send() and delay() methods  should look really familar by now. The same goes for the get_audio_info() and set_audio_info() methods on line 472 and line 489. The only thing you will see is that the set_audio_info() accepts 2 strings (a key and a value) from PHP. We talk about the set_metadata() function later, but the get_queue_length() function is also a simple one.

On line 613, you will see our structure filling up quite nicely with public methods  now :-)

Now, the only thing left to do now is talk about the set_metadata() function. This function is a bit different than the rest, since this function actually accepts an array. Internally in the libshout library, the metadata acts like an array. So from a PHP point of view, it makes sense to use arrays for this. So what we do on line 515 is accept an array. Then on line 520, we initialize a shout_metadata structure that will hold the data we want to set.

The for-loop on line 523 to 525 can been seen as a loop over all the elements from the array we just got. Then on line 533 we call  SEPARATE_ZVAL with the value for the current element. This separation actually creates a copy (but not really) of the current value so we can change it. If we leave this out, all changes to this value will also reflect back in the original PHP array, which we don’t want.

The convert_to_string makes sure that the value is always a string, which comes in handy since we need strings to store in our metadata structure.

Next on line 537, we try and get the key for the current value we are iterating. This key can either be a string or a number. If it’s a number, it means that the “str_key” value is not filled, and we will make sure that this number is converted into a string nevertheless. We do that on lines 538 to 541.

When all is done, we have a string with the key, and a string with the value (both not binary safe, but we still don’t care about that :)), and add it to our metadata structure through shout_metadata_add. The Z_STRVAL_PP is a macro that returns the string value for that current element.

When we are done with iterating, we can set the metadata on line 549, and afterwards we can free the metadata structure on line 550. We return the status code of the setting of the metadata on line 552.

And we’re done (for now)

That’s it really. All the functionality from libshout is implemented! I’ve added an example on how to use the extension on github as well (https://github.com/jaytaph/phpshout/blob/35ac2b32009936e53c3b336efed36a604d37d9f3/example/example.php), but make sure you use the latest version from https://github.com/jaytaph/phpshout/.

To test if it works, I’ve done the following. On my machine, I’ve setup an icecast server (I’m running OSX, so i’ve used a trail version of nicecast, but there are others). I’ve setup the server and changed this info in the example.php configuration (primarily the ip, port and password). Then, I added a file.mp3 to the directory so there is something to stream and started the example. If everything goes ok, you see a . appearing for every 4096kb of data that is stream to the server.

Next, I connected a few listeners (they connect to http://192.168.1.10:8000/listen.m3u, in my case) and dance away to my new PHP streamer!!  Oh the fun!

Conclusion

Cool things are cool, and writing something from scratch is one of them. Off course, there are hundreds better ways to stream data, but now it’s possible to do this from PHP as well. Lets stream your MP3 data, change the set_metadata according to the ID3 tags inside, show an upcoming playlist, a previous playlist, or even let people up/down vote for their favourite tunes and create a php jukebox. Who needs spotify or last.fm anymore? :p

Part 4: Populating our store

Let’s take a look at the source from this commit: https://github.com/jaytaph/phpshout/blob/8675c69c27675d4e1159a754a634f08fffc07746/shout.c

You see there is something changed in our store on line 40: there is a pointer to a shout structure. This structure is used by the shout library so every time we create a new object store, we should also initialize a shout object. You will see that things have changed in the constructor method on line 85. We fetch the actual store, and create a new shout object by issuing a shout_new(). If something goes wrong, and the shout object is NULL, we throw an exception.

Because we are adding things to our store, we must also make sure these structures gets freed when we don’t need them anymore. So in our shout_object_free function (line 225), we also free our shout structure by calling shout_free().

If you look closely to this file, you will see that something also has changed on line 352. Here we actually set the handler for cloning objects to NULL, meaning we can’t clone the object (try it, it will fail now). This is done because when we need to clone, we should also clone our internal data like the shout structure. Unfortunately, this data is very complex which means we need to copy over all the data, but also duplicating some of this data (because otherwise we end up changing data in two different places, for two different objects, which is the whole point of cloning to get rid of). So instead of dealing with this issue, I’ve decided to not allow cloning at all for the time being.

Also another thing that has changed, is that i’ve implemented a PHP_MSHUTDOWN function on line 386, and added this to the module entry structure on line 409. This will call the shout_shutdown(), to make sure things gets shutdown properly.

Part 5: Add getters.

A lot of the libshout functionality are actually based on getting and setting information, just like “normal” PHP getters and setters. This shout.c file actually has two ways of dealing with getting values which I will show you both.

The first method is the easiest to explain. Take a look at line 194, where there is a get_host PHP_METHOD.

Since getting stuff do not require any php parameters, the zend_parse_parameters_none() will check if there are any. If so, the php method will return false. On line 201, it will load the internal store. Line 202 will return a string value with the host as returned by the shout_get_host() function. This function needs a shout_t structure, which is the one we are saving in our store. From a PHP point of view, nobody needs to know anything about this shout_t structure so all this is done internally. As said, it will return a string with the current host.

Let’s take a quick look at setting info as well. Line 205 creates a set_host method. In this case, we DO need parameters, namely a string from PHP, which is the host we need to set. Line 210 will parse the parameters we get from PHP, makes sure we only get 1 string (that is the “s”), and this information is placed in the “host” and “host_len” variables. Remember I was talking about binary safe strings? The host is just a block of data which may or may not have \0 in them. The host_len is a integer that actually holds the length of the string. If we are really correct, we cannot assume that we can just use the “host” as a normal string because it isn’t. But for now this doesn’t matter, if you want to set a non-binary safe string as a host, that is perfectly fine, but just don’t think it will work that way:

$shout->set_host("Not a binary\x00 safe string");
print $shout->get_host();  #returns: "Not a binary"

Line 215 will actually set the host name by supplying our shout-t structure and the (non-binary-safe) host string. As a result, it will return a status code (the RET_* constants), which are LONG values, hence the RETURN_LONG. Notice that we don’t need to duplicate longs just like we need to do with RETURN_STRING.

Now, back to the getters, as you can see they are pretty easy to write, but in C, writing a function for each getter and for each setter would result in lots of duplicate code. There must be better ways to deal with this. And there is.

On line 95, I’ve added a (static) function called php_shout_get_handler_string. The argument list looks wierd, but again, it’s a ZEND macro:

static void php_shout_get_handler_string(INTERNAL_FUNCTION_PARAMETERS, const char *(*func)(shout_t *)) {

Part 3: create initial functionality.

Let’s take a look at the source from this commit: https://github.com/jaytaph/phpshout/blob/d9ad66512f23ff04ff2bb5822b833090d0a5d05b/shout.c

Here we actually created some simple functionality so we can use our shout-class in PHP (although it’s not doing very much).

We’ve added a php method on line 49. We use the PHP_METHOD macro for this, so php knows this is a method called “get_version” for the shout class. Internally it will create a function called zim_shout_get_version method since C isn’t OO or namespaced. This way, we can still distinguish our php methods (among other things). Line 50 will check if there are any parameters given to this method. If there are, it will return false since we do not accept any parameters.

Upon return on line 54, it will return the version of our shout library by calling shout_version, and returning this information as a PHP string. This is what the RETURN_STRING does, which is a macro that creates a PHP value in C the easy way. the “1″ parameter means that the string must be duplicated. Depending on your needs, this might be always the case since this will allow PHP to garbage collect unused variables.

So, this is a really easy function that does not much, but it shows you a bit on how “wrapping” a library works: we have a library function, we define a php function for that, we modify (optional) arguments from php so they can be fed into the library function and we return a result back to PHP. In fact, lot of PHP extensions work this way.

Now, let’s skip the constructor method on line 59 for now, and pay attention to line 148. Here you see we are filling our “shout_funcs” structure with the new get_version we just defined. we use ZEND_ACC_PUBLIC to tell that this function is a public function. So all that is left to do is create our PHP methods, add them to this structure and we’re done!

But there are some other things we can do, for instance creating class constants. This we do on line 240 to 261 (there are a lot of constants).

Again, PHP/Zend makes it really easy for use to create consants (strings or longs doesn’t matter). We just call zend_declare_class_constant_long, with our class_entry we just have saved ,the actual name and length of the class (the ERR_* constants for instance), and the actual value they represent. In our case, our constants are mapped to constants from the library file.

The ZEND_STRL macro:

Strings in PHP are binary safe. In C, they are not. Strings in C are normally terminated with a 0-byte, meaning that the string itself cannot hold a \0 value. And normally this is ok, exception when you are dealing with binary information in strings. So a lot of times, functions in PHP ask for a string PLUS the actual length of that string since it might not be possible for PHP itself to check the length by looking at the \0 byte. The ZEND_STRL macro returns the string PLUS the length of that string (assuming it’s NOT a binary string), so it’s easy to use in your code.

So in essence, these two functions are 100% similar:

test("hello world", strlen("hello world")-1);
test(ZEND_STRL("hello world");

Now, PHP has got lots of these nifty macro’s, and not everybody uses them, or even know them.

The constructor:

Let’s go back to our constructor method. On line 147, we see the constructor is added to our shout_funcs structure again. You will see that is has another flag namely ZEND_ACC_CTOR, which tells us this is a constructor. (I don’t know what happens when you place this flag onto anther function though). And the actual constructor method we define from line 59.

This looks a bit complicated with a lof of #if statements, but this is because things have changed in the past on how to handle things. This way, we can be sure that our extension can run on older versions of the zend-module-api (and thus older php versions) as well. There are actually more places where stuff like this needs to be done (when you want to support REALLY old PHP versions), but we can assume we don’t (since we are doing OO here).

We are basically saving and setting our own exception class, so when exceptions are thrown, it will be handled by our exception class. (I noticed there is a (big) bug in this code, which I let you try to find, while I will update my code :p). On line 78 we fetch our store object, something we will do a lot and store this in intern. It’s a common and easy way to get to our internal data, in case we need to.  For now, this line doesn’t serve any point since we don’t use the intern value anywhere.

On line 81, we are making a call to shout_init(), which initializes the shout library. There are some things we need to consider.

Right now, we are calling shout_init for EVERY object we are creating since it’s in it’s __construct(). However, this might not be what we want. We only want to initialize it once. So another option would be to place it in our MINIT function, which gets called only once. A perfect spot, but this means the shout_init() gets called, even for scripts that do not even use any shout functionality, which is something we don’t want either. For now, we don’t really care, since calling shout_init() multiple times isn’t a problem, but some library inits might have a problem with this. So make sure that when you call initialization functions, it’s done at the proper time.

Tying it all together:

Why not try and see if this thing actually works? :-)

1. Make sure you have got the libshout3-dev package installed.
2. Download the tarball for this blog-post: https://github.com/jaytaph/phpshout/tarball/blogpost-002
3. Untar this into a directory.
4. Run “phpize”.
5. Run “./configure”. If there is a complaint about not finding the library, install the library first or use “./configure –with-shout=<path>” with the path to the library (if it’s on /home/user/libshout/include/shout/shout.h”, use /home/usre/libshout as the path).
6. Run “make”, this will actually build the extension
7. Run “sudo make install”, (or make install as root) so the library will be installed.
8. Add the extension to your php.ini:   “extension=shout.so”
9. Check if the extension is working by issuing “php -m | grep shout”. This should show “shout”. You can also check phpinfo() on your webserver, but make sure the server has been restarted

Now you run your own scripts with shout capabilities (allthough very limited :p).

<?php

$shout = new Shout();
print "Our shout vesion: ".$shout->get_version()."\n";

which should output:

Our shout vesion: 2.2.2

In the next part, we will fill the rest of the functionality.

Part 1: download and read up the specs

There are two ways to create such a system. The first way is to create a 100% pure PHP class and the second one by creating a wrapper-extension for an existing (C) library. The biggest advantage of writing a PHP class is that  you don’t need to compile a new extension, but a big drawback is that you are implementing things in PHP that are already implemented in the C library. Furthermore, the PHP class will never be as fast as the C library. The drawback of writing an extension, is – well – you need to know a bit about C and the PHP toolkit with all the macro’s and quirks which might not be easy. However, you don’t need to duplicate functionality and nowadays compiling and installing an extension is pretty much automated.

So an extension it is, but before we can actually write the extension, we need to know how we would call the functionality from a PHP point of view. Do we create procedural function, should we create an OO interface? Or maybe even multiple interfaces? For this we need to know how the library works in C, which takes a bit of reading up. But luckily, there are some good documents for the library we are using: http://www.aelius.com/njh/libshout-doc/libshout.html and the actual libshout download has got some examples on how to use it.

The library itself is easy enough to install from a debian / ubuntu system, so I’d reckon CentOS would not be any different although I haven’t checked. A mere “apt-get install libshout3-dev” will do the trick in order to install the development files and the library itself. It’s not enough to get only the library, since we need to have the include files (*.h) files as well.

Part 2: create skeleton extension

Now for the fun part. I’ve decided to create a sort-of OO interface. Actually it’s has got not much to do with OO, but merely it’s a class that wraps our functionality. Might as well be a separate namespace, but this way it makes it easier to extend our class if you need to. Now, on to the actual creation of the extension. What I did was create a skeleton extension first. This skeleton holds not much info, but it’s easy to start from here.

You can find this at my GitHub commit: https://github.com/jaytaph/phpshout/tree/0fc0fe0fe3eb426591987194deff8bdecb717d34

This is pretty much the bare minimum for an (OO) extension. The most important files being config.m4 and shout.c.

config.m4:

If you are not using external libraries, your config.m4 doesn’t have to be this complex. This configuration “template” will be used to generate a configure script which in it’s turn creates a makefile to compile the actual extension. I’ve blogged about these configuration tools in another blog post here and there are still some in draft which I will release (very soon, I promise!). This config.m4 gives your ./configure script an option called “–with-shout”, where you can actually add the path to your shout.h include files. If you don’t specify it, it will check for /usr and /usr/local, but in case it’s somewhere else, you can easily specify it. That is basically what lines 5 to 25 do.

If this is found, lines 27 to 40 will check for the actual library called libshout(.so). This library should be in your LD_LIBRARY_PATH, which 9 out of 10 time is, so you don’t have to worry about it. If the library is found, it will do a sanity check if there is a function called shout_init() is defined. If so, we will assume you have to correct library. If not, it might be another libshout (either version, or maybe a complete other library).

shout.c:

Config.m4 should always be named config.m4, but shout.c is just the name of the extension, and might as well be named differently. For instance, when you have a “rainbow” extension (whatever that might be), the name of the main source file might be rainbow.c. Off course, your extension might need multiple source files (rainbow-pony.c, rainbow-unicorn.c etc), so you must add them to your config.m4 as well (on line 42).

Even though this file looks very complicated, it really isn’t.

Line 34 includes the shout library include file. This allows us to use the shout functions. Because we searched for this include file through the config.m4 and the configure script, the makefile automatically knows where this file is located so we don’t have to worry about it anymore. Line 38-40 defines an internal structure that holds all information for a single PHP shout-object, sometimes called the “object store”. So when we have 2 objects in PHP, we have two php_shout_obj structures or “stores”. For now, this only include the actual object information (the zend_object value), but later on we can fill it with internal data. You can see this a bit of the private values of our class if that makes sense to you.

Next up are some (static) functions for creating, freeing, and cloning our objects. These are standard boilerplate stuff. For instance the shout_object_free() function does nothing more than destroying our internal object through a zend_object_std_dtor() call, and free()’ing our store.

The shout_object_new_ex() and shout_object_new() are functions to create a new object store. The shout_object_new is nothing more than a redirect to shout_object_new_ex(). Lines 68 and 69 initializes some memory for our internal php_shout_object structure and clearing this memory. lines 75 and 76 initializes the internal zend_object in our structure with the default properties.

Lines 78 and 79 will set the handlers (functions) for this object, basically which function it will need to call when the object will be freed and the basic handlers (which are saved in shout_object_handlers, we set these values later). After that, we must return a zend_object_value object with this information.

Line 91 to 101 will deal with cloning an object. This function gets the old object store, creates a new object store (line 94) and line 96 copies/duplicate the actual zend_object data that is located in our store. If there is any other internal data in our store, we need to make sure we duplicate or copy this over. But since we don’t have anything stored yet, we skip this part for now.

Those are the house-keeping functions, but somehow this all must be connected together. This is where the next functions are structure come in. On line 105, we define a structure that holds all the functions as seen from our PHP class. You can see, we don’t have any and that’s ok, since this is a template.

Next up on line 112, the PHP_MINIT_FUNCTION. This is a macro (as you will see a lot in the PHP core and extensions), which defines a module initialization function for the “shout” extension. This function will copy the standard handlers that php uses into it’s own shout_object_handlers (line 117). This way, when we want handle something differently in our own shout-classes, we can easily do so. Remember that these handlers are copied over every time we create a new object (line 79).

The next lines – 122 creates a new class entry with our functions (which we don’t have yet). Line 123 tells it that we need to use shout_object_new() function in order to create new objects and line 124 sets our clone() handler to be our own customized clone-handler. Line 126 actually registers our class and we get our own shout class entry back which we save for later usage.

Lines 134 to 140 is a function that will define the information that is shown during php’s phpinfo(). This will print a table that shout support is enabled (obviously, otherwise this whole table would not be present), plus the version of our php extension plus the version of the library we are using (you see here we are using our first call to the actual shout library!).

Now, in order to connect all this together to PHP can actually use it, we have to create a zend_module_entry structure on line 144. It consists of standard module headers (STANDARD_MODULE_HEADER, the name of the extension (shout), the functions used (shout_funcs), our module startup function that will be run ONCE the extension is loaded, the module shutdown function that will be run ONCE the extension is unloaded (in our case, we haven’t defined any, so this is NULL). The next two NULL values are the functions loaded every time a request is made. Then the function that returns the information for phpinfo(), and last the version of our module, plus some module properties.

TSRMLS_CC?

On occasion, you will see argument lists ended with TSRMLS_CC. If you don’t know PHP, these doesn’t make any sense, since they are there for thread-safety. For more information about it, take a look at the excellent post about PHP and thread safety: http://blog.golemon.com/2006/06/what-heck-is-tsrmlscc-anyway.html

Next, up, we need to implement functionality.

So, if you like to contribute you can use the following workflow for now. The most up-to-date workflow should be available on the php.net wiki at https://wiki.php.net/vcs/gitworkflow (the workflow for external contributors section). Make sure you follow those directions if they do not match with these…

Step 1: fork the php-src repo

Assuming you already have created a github account, you must clone the php source repository to your own github account. This can be done by going to the following url: https://github.com/php/php-src, and click on the “fork” button in the top right. This will create a fork of the php source into your own github account.

Step 2: clone your own repository.

Once you have created the repository, you can clone it onto your development machine.

git clone git@github.com:<yourname>/php-src.git php-src

This will take a while, since the repo is around 100MB in size. Just let git do his work for a while.

Step 3: add a remote branch.

Next up, you must add a remote branch. Basically, what you are doing is creating a link between your repository on your development computer and the php-src “master” repository (not your github repository, but from php itself). This is called the “upstream”, and that’s why we name it upstream. Your own github repository is usually called “origin”.

cd php-src
git remote add upstream https://github.com/php/php-src.git

When you check the remotes with the command “git remote -v”, you can actually see both the origin and upstream remotes are available. In my case:

jthijssen@debian-jth:~/php$ git remote -v
origin	git@github.com:jaytaph/php-src.git (fetch)
origin	git@github.com:jaytaph/php-src.git (push)
upstream	https://github.com/php/php-src.git (fetch)
upstream	https://github.com/php/php-src.git (push)

Step 4: create a new (tracking) branch.

Now we can actually create a new “branch” in which we are going to work. Every feature should have its own branch. When you have a issues a bug inside the bug database for PHP, use that number as the branch name:

git branch --track issue-<issuenr> origin/PHP-5.4

Now, a few things are important: first of all, PHP is using tracking-branches, which works a little bit different than local branches. You can read more about that here: http://gitready.com/beginner/2009/03/09/remote-tracking-branches.html.

Secondly, the “issue-<issuenr>” is the branch-name we are using. It might as well have been named “cool-new-awesome-feature”, but try to keep it as readable as possible. You are doing your mergers a big favor here.

Another important thing: we are creating an issue for the PHP-5.4 branch here. If you have a feature that isn’t ready for 5.4, or something that needs to be inside 5.3 (like a security fix that also need to be inside 5.4 and the “master” (or trunk as it used to be called)), always use the lowest version. In that case, it would be PHP-5.3. If you don’t know, ask, or use the master branch.

Step 5: switch to the new branch.

Now, we must switch to the new branch, so we can actually work in it:

git checkout issue-<issuenr>

When you look with “git branch”, you will see your (local) branches, and with “git branch -v” you will see some more info:

jthijssen@debian-jth:~/php$ git branch -v
* issue-60742 91f2d38 [ahead 1] Issue-60742: Added FilesystemIterator::OTHER_MODE_MASK
  master      dea5376 Merge branch 'PHP-5.4'

The * in front shows you which is the current branch you are working on. It probably has green color as well.

Step 6: do your stuff.

Now it’s time for the fun part: fixing the code, adding it and committing it.

Every time you want to create a commit, just do a “git add” on the files, following by a “git commit” and enter a log message. Don’t worry about over-committing. That actually doesn’t hurt, under-committing does.

Step 7: rebase your commits in case new commits have been added (optional)

It might be possible that others have committed new stuff to the branch you like to push to. This would be visible in the “upstream” branch, but not in your “origin” branch. There we rebase our commits, basically telling the system to cut away all our commit, update to the latest upstream version and paste our commits to the end. We will do this with the following:

git pull --rebase upstream PHP-5.4

Make sure here that PHP-5.4 is actually the branch name you originated your feature branch from (so it could be PHP-5.3 or probably master as well).

Note though, that rebasing is one of the most difficult things to get right and understand when it comes to git. If you don’t get it working properly, that’s fine and just skip this step. The php developers can just merge your request with others instead, just like in the “old” svn way.

Step 8: push your commits of the new branch to your github

Now we issue a:

git push origin issue-<issuenr>

This will push our new branch with all the commits to our github repository. When this is done, you can see the new branch on your github account, but you probably have to look for it in the “branches”:

Step 9: issue a pull request

Now for the most rewarding part: we can finally issue a pull request to the PHP developers. First, switch to the branch you just have pushed to your repository and click on the “pull request” button next to the original “fork” button. You will get a screen like this:

The only thing you need to make sure is that the “base branch” is set to the correct branch. In this case it says “php/php-src @ master”, but it should have been “php/php-src @ PHP-5.4″. Somehow this is not set correctly by default. You need to change this by typing in the correct name, and press “Update Commit Range” button. If everything goes to plan, you will see the number of commits and the actual files that have been changed.

Once done checking, you can press the big green button that submits the pull request to the PHP and hope for the best!

When I started blogging I did probably the same as anybody else: setup WordPress, added 2 blog-posts, did some Google Analytics setup and basically waited until anyone else than my mom visited the site. It didn’t happen..  So, after those two blog post, probably 95% of the users would just quit blogging and continue with their daily lives.

I sorta didn’t..

What happened, is that I started to blog more and started to realize the reason why I actually blogged: it was not show off to everyone else on how smart I am on certain subjects (which I am :P), but rather to figure out if I’m actually comprehending the matter where I’m blogging about. You see, it’s easy to read up some documentation, check some blogs and articles about a subject, and call yourself an expert. People tend to do that a lot. But what happens is that when you write a blog-post about a subject, you cannot write anything unless you are 100% clear about the subject. I try the same strategy when I’m teaching stuff: the best way for me to know if you understand the matter, is to let you explain it to me, not the other way around.

If you don’t know the matter well enough, then there is no possible way you can write a blog-post on it (or do a talk on the subject, or teaching it). So writing blog-posts for me was a “final test” to figure out if really know the stuff I think I know. And it turns out that a lot of times I didn’t.

And this actually has two advantages: first, you don’t get frustrated by people not viewing your blog, since this is not your goal anymore and B) because your are not blogging for others,it’s easy enough to write about anything that pops inside your mind. And it turns out that writing blog-posts get easier and easier over time. I have a lot of subjects in draft which I either need to finish, or cannot yet release, but a lot of posts are just about problems I’m trying to solve or just solved, or things that annoy me where I blog about. You can write a blog-post in like 10 minutes without problem, which isn’t much more than writing a (way too long) email.

This actually has led that my blog grew with more and more articles about a very broad range of subjects. And just like a snowball effect, a blog-post got picked up by somebody, and more people got onto the site, and even more, and all of sudden you don’t even have to subject your blog-posts to the “major” outlets, because they are automatically interested in the things you are blogging about.

So the best advice I can give to you: don’t blog for others, blog for yourself. It’s a good way to archive the things you know and a great way to find things back you forget in a few months. And after a while, people WILL visit your blog. And funny enough, just this morning I was talking on Freenode with somebody about a subject that was one of my first blog-posts I written on this site.

So,.. in the end, my ideas of march pledge will not be about blogging more,  it will be about blogging just in the same speed I am doing now. This month, and the next months as well.

Now, at first glance, this seems a valid reason: if you don’t know which port to attack, you can’t attack it at all :-). But if you re-read that last line, you will notice this is nothing more than security through obscurity, and if you think that that is a good idea, you should not be allowed behind a computer.. ever…

But there are more reasons why this is a bad idea and one of the most important reason has to do with a bit of the (Linux) way of handling TCP/IP ports. When you are logged onto a system as a non-root user (anyone not being uid 0), you cannot create a listing TCP or UDP port below 1024. This is because port numbers below 1024 are so-called privileged ports and can only be opened by root or processes that are running as root. So for instance, when your webserver (apache, nginx etc) will start, it will do so as the privileged root user in order to open up a listening connection to port 80 (the port that by default will be used for HTTP traffic). Now, as soon as the port is opened and everything that needs to be done as root is done, the webserver will fall back to a non-privileged user (either the www-data, apache, or nobody user). From that point, when something bad is happening, it is only limited to the rights that that user has.

Now, back to SSH: when we start SSH on port 22, we know for a fact that this is done by root or a root-process since no other user could possibly open that port. But what happens when we move SSH to port 2222? This port can be opened without a privileged account, which means I can write a simple script that listens to port 2222 and mimics SSH in order to capture your passwords. And this can easily be done with simple tools commonly available on every linux system/server. So running SSH on a non-privileged port makes it potentially LESS secure, not MORE. You have no way of knowing if you are talking to the real SSH server or not. This reason, and this reason alone makes it that you should NEVER EVER use a non-privileged port for running your SSH server.

On to the next reason not to change ports: A lot of applications actually EXPECT ssh traffic on port 22. Now this might be a debate wether or not those programs are developed properly, but it’s a fact. Even though you can easily change the port in many applications but not all of them do. Trust me, it WILL be annoying for developers, sysadmins and users to operate on your SSH-port 52241, especially since they are using 20 boxes, each with a different SSH port.

Another issue: many corporations have incoming and outgoing firewalls, meaning you cannot goto any site to any random port and expect it to work. Some secure ports, like port 22 are often exempt from that, while other ports like port 25 or 110 are blocked.

Now, let’s pretend you STILL want to move the port away because you get so many attacks on your SSH port. First of all: are you able to logon as root? If so, fix that now. Secondly: are you using passwords? If so, fix that now and change into public key authentication. But think about it: is it a problem to have so many people banging at the front of your house? They are just there testing your defenses! :-) I’d rather have an angry mob outside my house knowing it would mean I would do everything to make sure they can’t break into my house, than to have an open door, that I do not know about, until some lonely passant just drops in to say hi (and trash the place, or worse).

But if you are REALLY worried about that angry mob, there are better ways to hide the door than to move it. Try port-knocking. Again, security through obscurity, but if you must, this is probably the only viable way to do it. There are several ways to implement port-knocking. There are 3rd party tools, which are A) way to big and complex and B) are implemented in userland so don’t use them. Instead, Iptables has got a very nifty module called “recent”, which allows you to create simple – yet effective – port knocking sequences.

Take a look at the following example:

${IPTABLES} -A INPUT -p tcp --dport 3456 -m recent --set --name portknock
${IPTABLES} -A INPUT -p tcp --syn --dport 22 -m recent --rcheck \
   --seconds 60 --name portknock -j ACCEPT
${IPTABLES} -A INPUT -p tcp --syn --dport 22 -j DENY

What this does, is that as soon as something tries to connect to port 3456 (yes, a non-privileged port, but no problem, nothing is running on it), it will set a flag called “portknock”. Now, when we try to setup a connection to the SSH port, it will check to see if your IP has set a “portknock” flag during the last 60 seconds. If not, it will not accept the connection. And the third line will by default deny any access to SSH together as a failsafe. (I’ve heard before that it would be hard for users to remember that they need to connect to another port first, but somebody who cannot click a link, type a url, or telnet to a port, should he or she really have SSH access to begin with???).

Now, don’t directly copy/paste this into your firewall, but you can see how easy it is to “hide” your ports. You can even create a whole serie of knock sequences if you like but this will only annoy your users and gains you nothing.

Again, this does not change ANYTHING on the fact that it should not matter if your port is out in the open or not. Don’t fall for security through obscurity, because that will probably be the easiest way to get your box hacked. And please, as a hosting company, a system administrator or even a developer, don’t ever tell others that moving your SSH port away is a good thing. Because it’s not, and it never will be.