Tip #13: Understand the difference between requests and impressions

If you have read The Guru’s article on how the banner delivery process works in the OpenX ad server, then you may have noticed that when a banner is delivered, two items of data are logged — a request, and an impression1.

Understanding the difference between requests and impressions is important, and once understood, can provide you with some interesting insight into your OpenX banner delivery performance.

Requests and impressions in the OpenX database

Before diving into the difference between requests and impressions, it’s useful to understand where requests and impressions are logged.

If you are running OpenX 2.4 or OpenX 2.6, then requests are logged into the “data_raw_ad_request” table2, while impressions are logged into the “data_raw_ad_impression” table.

If you are running OpenX 2.8, then requests are logged into the “data_bkt_r” table, while impressions are logged into the “data_bkt_m” table.

If you have never done so before, log into your OpenX database now, and take a quick look at the contents of these tables — but be careful! These tables, especially in the older OpenX 2.4 and OpenX 2.6 versions, can become very large, so running SQL commands on these tables may take a very long time and may even impact the performance of your server. As a result, it is generally worth limiting the number of rows you look at when you inspect these tables. For example, you might use the following commands3:

OpenX 2.4 & OpenX 2.6

SELECT * FROM data_raw_ad_request ORDER BY date_time DESC LIMIT 20;

SELECT * FROM data_raw_ad_impression ORDER BY date_time DESC LIMIT 20;

OpenX 2.8

SELECT * FROM data_bkt_r ORDER BY interval_start DESC LIMIT 20;

SELECT * FROM data_bkt_m ORDER BY interval_start DESC LIMIT 20;

You should see some of your logged request and impression data.

If you are not seeing any request or impression data at all, have you delivered any banners recently? Try delivering some banners, and see if you can now see requests and impressions logged.

If you see impression data, but no request data, and you are using OpenX 2.4 or OpenX 2.6, do you have request logging enabled? Check:

  • In OpenX 2.4: While logged in to OpenX as the administrator account, go to Settings, Main Settings, Statistics and Maintenance Settings, and check if the “Log an Ad Request every time an advertisement is requested” setting is enabled.
  • In OpenX 2.6: While “working as” the “Administrator account”4, go to My Account, Global Settings, Banner Logging Settings, and check if the “Log a request every time a banner is requested” setting is enabled.

Please be aware, however, that as OpenX 2.4 and OpenX 2.6 don’t have the newer, faster logging system that was introduced in OpenX 2.8, enabling request logging on an OpenX 2.4 or OpenX 2.6 system is likely to add significant additional load to your server. Think carefully before deciding to enabling this setting, and ensure that you monitor your OpenX server carefully after to make sure that the server’s performance is still okay.

Requests and impressions in the OpenX user interface

Now that you’ve seen the request and impression data in the OpenX database, and confirmed that it is being logged, it makes sense to ensure that this data can be seen in the OpenX user interface. By default, impression data will be viewable in the OpenX user interface, but request data is not viewable. To enable the ability to view request data in the user interface:

  • In OpenX 2.4: While logged in to OpenX as the administrator account, go to Settings, Main Settings, Interface Defaults. Here you find a list of the various columns OpenX can display in the user interface’s various statistics screens. If it is not enabled, and you would like view requests in the statistics screens, check the “Show Requests column” setting for those account types you would like to be able to see requests.
  • In OpenX 2.6: While “working as” the “Administrator account”5, go to My Account, Account Preferences, User Interface Preferences. Here you will find a list of the various columns OpenX can display in the user interface’s various statistics screens. If it is not enabled, and you would like to view requests in the statistics screens, check the “Requests” setting. You can also change the order in which the viewable columns are displayed by changing the “Column Rank” numbers.
  • In OpenX 2.8: While “working as” the “Administrator account”6, go to My Account, Preferences, User Interface Preferences. Here you will find a list of the various columns OpenX can display in the user interface’s various statistics screens. If it is not enabled, and you would like to view requests in the statistics screens, check the “Requests” setting. You can also change the order in which the viewable columns are displayed by changing the “Column Rank” numbers.

Note that while OpenX 2.4 provides a facility for the administrator account to control which columns the four account types that exist in OpenX 2.4 can see, this is not possible in OpenX 2.6 and OpenX 2.8, where every account can modify which columns they want to view themselves — the above instructions simply change the master default account preferences for the “Administrator account” which all other accounts inherit their preferences from.

So, what’s the difference, and why do I care?

If you read the article on the banner delivery process carefully, you will note that a request is logged as soon as the OpenX ad server has determined which banner it is going to display. However, an impression is not logged until after the banner has actually been displayed7.

You will notice, if you are logging requests and impressions, and displaying both columns in the user interface, that as a general rule, the number of requests are greater than the number of impressions. The difference between requests and impressions is generally referred to as the drop off rate. A “high” drop off rate is bad for two reasons:

  1. If your OpenX server is performing work selecting banners that are never shown, then your server is doing work that is irrelevant — so, your server performance is less than it could be, which is costing you money.
  2. If users are viewing your web site, and requesting banners, but the banners are not being displayed, then you are losing out on income — regardless of if your campaigns are paying you on a CPM, CPC or CPA basis, if the banners don’t show up, you won’t get paid.

So, what is a “high” drop off rate? There is no industry defined standard that The Guru is aware of, but anything more than 10% would certainly be cause for concern, and even drop off rates lower than this may represent a significant reduction of potential income for some sites. The only real rule here is that the lowest possible achievable drop off rate is desirable.

How do I fix a bad drop off rate?

This, of course, is the big question. The way to fix a bad drop off rate is to identify why banners are being selected for display, but are not actually being displayed, and to then act accordingly. Here are some of the top reasons for banners failing to display, and suggestions on how to address these issues:

Incorrect use of the Single Page Call tag

The Single Page Call tag calls OpenX just once, to obtain a banner for all of the zones that are to be displayed on a page. By default, the Single Page Call tag selects a banner for all of the zones in a website. So, if only some of the zones are going to be displayed on a page, then the tag needs to be edited before it is used, to ensure that banners are not selected that will never be displayed.

Large banner file sizes

If the file size of a banner is very large (e.g. large flash banners, etc.) then users with slower internet connections may “navigate away” from a page before the banner has finished downloading to their computer. Try to avoid using having very large banner file sizes, or, if this is essential, consider purchasing the MaxMind NetSpeed database, which will allow you to target large file size banners to users with faster internet connections only, while targeting alternative, smaller file size banners to those on dial-up connections.

Slow 3rd party banners

Many OpenX users will generate a large part of their income from ad networks, who supply an HTML tag that loads banners from the ad networks’ servers. It is an unfortunate fact of life that sometimes, the 3rd parties’ servers are not always as fast as would be desired. The only real option here is to monitor the drop off rate at the banner level in the statistics pages, and see if there are particular ad networks that are causing you persistent problems — and if there are, consider the income they generate. If they are not a major part of your income, perhaps dropping these ad networks may result in overall better income for your site.

Slow websites and poor banner placement

If your website itself is slow to load, then as with large banners, users can sometimes “navigate away” from a page before the page has finished loading. In these cases, banners that are placed towards the bottom of the page may not get a chance to complete loading before the user goes elsewhere. Upgrading your server to ensure your web site loads quickly at all times will help, as may re-considering your website design and banner placement.

  1. OpenX 2.8 introduced request and impression logging by default. In OpenX 2.6 and 2.4, request logging was optional, and not enabled by default. []
  2. If request logging is enabled. []
  3. You may need to edit these example commands, if your OpenX installation uses a database table prefix, e.g. “phpads_”, “oa_”, “ox_”, etc. []
  4. You will need to be logged into OpenX as a user that has permission to act as the “Administrator account”. []
  5. As before, you will need to be logged into OpenX as a user that has permission to act as the “Administrator account”. []
  6. As before, you will need to be logged into OpenX as a user that has permission to act as the “Administrator account”. []
  7. There is one exception to this rule, which results in the request and the impression being logged at the same time. As mentioned in the banner delivery process article, this is when an Image, No Cookie Image or XML-RPC zone tag is used to deliver a banner, because these tags do not support the logging of impressions via a logging beacon. []

12 comments to Tip #13: Understand the difference between requests and impressions

  • su

    Thanks for the article. I have a question. In my openx server, the request number is greater than impression number. What can be the reason? Could it be caused by page cache?
    Req. Impr.
    16:00 – 16:59 32 72
    17:00 – 17:59 32 71

  • The first thing to check is if you are using the Single Page Call tag correctly, assuming you are using SPC. If you are not, then you will get many more requests than impressions.

    Alternatively, the difference between requests and impressions tends to simply be a matter of users “navigating away” from your page before the banners complete loading. Look into measuring your server performance to make sure OpenX is not running slowly!

    The issue of “page caching” tends to result in more impressions than requests, rather than the other way around!

  • su

    Thanks for your quick response.
    Sorry, I actually meant my impression number is greater than the request number and I wanted to know the reason. It’s totally my fault to type it wrongly.

    In my test ad page, I did implement a ‘page caching’ with a servlet filter (by checking if-modified-since header…, and send a 304 response if page is not modified), so that’s the cause of getting more impressions than requests? Is there a way to get accurate request number in this case?

    • Ah, if impressions are greater than requests, the issue tends to stem from the fact that requests are logged by OpenX during the process of interpreting the zone tag, but impressions are logged by a straight up beacon call that is returned by OpenX to log the impression. This tends to result in the tag getting called directly (either as a result of spiders or caching), and so you end up with more impressions being logged.

      Unfortunately, there’s not much you can do about it, as OpenX doesn’t have functionality (yet?) to ensure that an impression is only logged if it follows on from a request…

  • su

    Thanks, your explanation helps!

  • [...] 英文原文地址:http://www.openxtips.com/2009/05/tip-13-understand-the-difference-between-requests-and-impressions/ [...]

  • Is there a big performance hit to OpenX for logging requests?

    • Hi Mark!

      The answer depends entirely on where the bottlenecks are in your system — if indeed there are any!

      If, for example, you have a low CPU powered machine, then you might find that OpenX is CPU-bound, and so adding request logging will make no difference at all — as your install is limited to a maximum number of impressions/second by the amount of CPU power you have, then logging extra data to the database won’t change that.

      However, if you have a more powerful server, and the disk I/O performance is in fact your bottleneck (that is, the maximum number of impressions/second you can serve is limited by how fast you can write the impression data to the database), then turning on request logging will in theory HALVE the performance of you server, as you are now DOUBLING the amount of data you need to write to the database for every impression (i.e. you now need to log a request and an impression, where before you just logged an impression).

      Hope that helps!

  • abbas

    Thanx for the article, I have one swf file in which different images load and i want to increase the impression after loading each image.
    can i do impression increment manually means with calling the ip?
    for example through
    http://mysite/openx/www/delivery/lg.php?bannerid=11&campaignid=4&zoneid=7&OACCAP=500&OASCCAP=500&loc=http%3A%2F%2Fmysite%2Fmultizone.html&cb=eda854695d

    Is it the correct way for doing it or there is any other better approach to achieve this thing

    • Hi abbas,

      Yes, calling the log script manually should do the trick, provided you don’t have OpenX set up to ignore repeat impressions that occur within a certain time period of previous impressions (if your version of OpenX supports this; not all versions do).

  • abbas

    Thanx for your reply I want to know two more things
    1) How i can add manually clicks against the banner.
    2) When i create the banner then in which tables that banner information store (Suppose my banner is external banner).

    • Hi abbas,

      You can’t manually ad clicks against banners without actually calling the click script – there’s no “log” script like there is for impressions. So, you either need to call that script, or add the data to the database manually. Of course, you shouldn’t do this, because you only want to track real clicks, right? :-)

      When you add a banner to OpenX, the data for that banner is stored in the “banners” table.