Tip #16: Measure performance

Here’s a great question from reader Komson, who writes:

Dear Guru,

We are using OpenX, and we are facing a problem — the server “goes down” at peak times.

Can you please shed some light on the maximum number of impression per hour, in general, you recommend you should not go over?

e.g. you should not serve over 150,000 impressions per hour.

Thanks in advance!

Here’s a question in response:

I’m planning a party. I’ve already bought some food. How many people can I invite?

It’s a somewhat flippant question, for sure, but hopefully you get the idea — it’s not possible to answer the question of how many people can come to the party without knowing more about the situation. To be able to accurately determine how many people can be invited to the party, you will need to know:

  • How hungry do you expect the guests to be? Are they going to be happy getting just a few snacks, or will they will be expecting a full meal?; and
  • How much food do you have for the party?

The same thing goes for answering the question about how many impressions per hour you can serve with OpenX.

First up, there’s the question of how much you want to “feed” to your guests. You may be aware that in computing, there are two different ways of measuring “performance”. There’s throughput1, and response time2.

Obviously, users of your web site won’t care about throughput3 — all they care about is response time, because they don’t want to sit around waiting for ages while your website loads. They want it to load now.

So, this is the question of how “hungry” your guests are — they want to be fed, and chances are they want to be fed now, but how much food do you need to ensure all of your guests are well fed? In this case, though, given that you’ve already bought the food (i.e. you’ve already got OpenX installed and set up ready to go on a server you’ve bought, or a hosted service you’ve paid for), its more useful to ask: how many guests can you invite to the party and still ensure everyone gets a good feed? That is, how many impressions per hour can you serve, and still ensure that your website loads quickly?

Of course, if we were really talking about food that you had bought for a party, then once you’ve decided how much you think people will want to eat, it would be pretty obvious by just looking at it how much food you have as to how many people you would be able to feed with it. Naturally, things aren’t quite so simple with an OpenX installation. How much “food” do you really have, so you can figure out how many people can come? That is, once you’ve decided what the maximum average response time you want your users to experience is, how many impressions per hour can you serve with your existing hardware before you will start having a response time that is worse than this?4

Unfortunately, there’s only one way to figure this out — you have to measure it!

Luckily, OpenX have actually provided some scripts5 to take some of the pain out of measuring your ad server performance, but unfortunately, in the end, if you want to really know how many impressions per hour your OpenX installation can serve, while still keeping people happy, then unfortunately, you have to do some work.

Finally, before you dive into the task of measuring performance (or, if you’ve done so already, and you’re not happy with the results), don’t forget that the OpenX blog has three different articles that all relate to performance — these are definitely worth reading, especially if you have an OpenX ad server installation that is having performance issues!

  1. Throughput is a measure of the number of units of work that can be performed per unit time. So, in this case, it is the number of impressions per hour that your OpenX installation can serve. []
  2. Response time is a measure of how long it takes to get a response — that is, how long does it take from the moment a user requests a banner from your OpenX installation until the banner is displayed in the user’s web browser. []
  3. You obviously care about throughput, as this is the whole point of the original question — how many impressions per hour can you serve from your OpenX installation? However, your can be assured that your website users won’t care about this. :-) []
  4. If you want to get really technical, you can also consider what the maximum standard deviation you will allow as well, to address the case of those response times that are greater than the average response time. Think of the standard deviation of the average response time as a measure of that annoying thing that happens at a party where, although there is plenty of food, the people who are standing nearest to the kitchen eat it all, while people furthest away from the kitchen go hungry while they stand around waiting for the plates of food to get passed to them. The greater the standard deviation, the more hungry people will be getting angry in the far corner of your party. []
  5. The OpenX scripts page indicates that they are for testing new installations of OpenX before they are put into production. While this is obviously the ideal case, it’s still possible to do performance testing on a live installation — but do the performance testing off-peak (e.g. the middle of night) so that you won’t have many users looking at your website if you end up pushing your server too hard, and you have really bad response times during testing. []

Tip #12: Check out the EsDev icon set

Check out the free replacement icon set for OpenX 2.8 from EsDeveloped.

The Guru has, and it’s rather good!

Tip #6: Backup your database, code & banners

It might seem strange to you that The Guru would write a tip for OpenX that is as seemingly simple as “backup your database, code and banner files.”

However, you would not believe the number of times The Guru has been contacted by a client, asking for help with their ad server installation, because something has gone wrong with their server hardware, or because an upgrade failed, and then, on investigation, has found that there’s been no effort made at all to ensure that the OpenX database, OpenX code base (including the configuration files) and banner files, if they have been stored externally to OpenX, are all safely backed up.

In these situations, there is little than can be done if the disaster is catastrophic enough, other than to get a new installation of OpenX up and running, and for the client to start putting in all of their websites, zones, advertisers, campaigns, banners and channels from scratch…

If you rely on OpenX to manage your banners, and you can think of more productive things to do with your time than race against the clock to get OpenX set up from scratch before your advertisers take their money elsewhere, then please, heed the warning: Make regular backups of your OpenX database, OpenX code base and, if necessary, your banner files – especially before attempting an upgrade!

Tip #5: Store local banner (website) files outside of OpenX

Using the OpenX local banner (website) image banner type? If so, don’t use the default storage location (i.e. the OpenX “www/images” directory). If you do use this location, then every time you upgrade OpenX, you’ll need to remember to copy your creative files over to the new OpenX code base.

Instead, create a new directory for your creative files to be stored in and served from. Make sure that this directory is outside of the OpenX code base. (Or, if you have a really large OpenX installation, consider setting up an FTP server that can be used to replicate your creative files to all of your delivery servers, and use the FTP server as your “storage location”.)

This way, you’ll never have to copy your creative files on upgrade ever again!

Tip #4: Synchronize your clocks

It doesn’t matter if you are running OpenX on a single server, or if you are running OpenX on 20 (or more!) servers. Either way, you probably want the statistics collected and displayed by OpenX to be as accurate as they possibly can be.

One very simple thing that you can do to help ensure your statistics are accurate is to ensure that on all servers where you run OpenX, you have correctly configured the system time and timezone, and that you are running a tool to ensure that your clock is synchronized with a master time source. (Clock synchronization is clearly important when running OpenX across a number of servers, but it certainly doesn’t hurt on a single server setup either!)

This is something that is very easy to do, both for Linux and Windows systems, and yet is something which The Guru very rarely sees set up. Why not update your OpenX servers today?