Simplifying Hit Calculations for Xignite Web Services
First Published Friday, 4th March 2011 11:54 am from Xand : Joel York
The opinions expressed by this blogger and those providing comments are theirs alone, this does not reflect the opinion of Automated Trader or any employee thereof. Automated Trader is not responsible for the accuracy of any of the information supplied by this article.
/>
src="http://api.tweetmeme.com/imagebutton.gif?url=http%3A%2F%2Fxignite.web-services-blog.com%2F2011%2F02%2Fsimplifying-hit-calculations-for-xignite-web-services%2F&source=xignite&style=normal&b=2"
height="61" width="50" />
src="http://xignite.web-services-blog.com/media/easy.png"
alt="xignite hits easy" />At Xignite, most of our Web service
subscription plans are quoted according to a uniform usage metric
known as "hits." Estimating the number of hits you need for your
application is straightforward, once you get the basic idea.
However, for the first time customer it can be a little cryptic.
So, we're providing this blog series to help you through the
calculation process.
The first post in this
blog series provided a comprehensive href="http://xignite.web-services-blog.com/2011/02/hit-calculation-tutorial-for-xignite-web-services/"
target="_blank">hit calculation tutorial and
provided a href="http://xignite.web-services-blog.com/media/xignite-hit-estimation-worksheet.xlsx"
target="_blank">hit calculation spreadsheet that
covers all usage scenarios and introduced the following formula
for calculating monthly hits:
Monthly Hits =
Hits per Data Block
x Request Population x Request Frequency x 1
month
It's a great general
reference, however, the vast majority of applications fall into a
few standard usage scenarios where it's possible to dramatically
simplify this formula. This second post in the series surveys
these standard usage scenarios and guides you through the process
of simplifying the hit calculation to something you actually can
do on the back of an envelope.
It Doesn't
Need to Be Perfect
First, here's a word to
the wise. Xignite subscription plans span an enormous range of
usage scenarios, from 600 hits up to 60,000,000 hits and beyond.
For most Xignite Web services, each subscription plan is
separated by a 10X factor in usage, i.e., 6K, 60K, 600K, 6M, etc.
That's a huge step between each level (and a huge volume
discount)! Therefore, your hit calculations don't need to be
anything close to perfect. And, if it turns out your usage is
different from the original estimate, you can simply change your
subscription plan when it comes up for renewal. Hit estimates
only require back-of-the-envelope estimates that are accurate
within a factor of 10X and can be done in minutes, so you can
decide which subscription plan fits your needs. The tips that
follow will help you do just that.
Identify
the Request Frequency Scenario
For most
applications, the request frequency is the single most important
factor in determining the subscription plan required.
While the number of hits per data block
may vary from 1 to 100 (e.g., single option price vs. a long
option chain), and the request population may vary from 1 to
10,000 (e.g., single stock symbol vs. the total stocks on an
exchange), the request frequency can easily vary from 1 to
10,000,000 (e.g., monthly downloads to 4 times per second cache
refreshes). However, this huge range of request frequencies can
be reduced to a handful of common trigger event
scenarios.
• End User Page Views
/> • Transactions Processed
•
Real-time Cache Refreshes
• End-of-Day
Historical Data Updates
• Ad-hoc
Downloads
Zeroing in on the event that
triggers a Web service request allows you to simplify the general
formula above into a natural format that relates directly to the
usage pattern of your application. Calculating the request
frequency is then a matter of estimating the average number of
trigger events that occur per month as follows.
Monthly Hits =
Hits per Data Block
x Request Population x Page Views per Month
(or
Transactions per Month, Cache Refreshes per Month,
etc.)
Real-time cache refreshes
usually generate the highest request frequency. For example, a
cache of real-time quotes that updates every second, 8 hours a
day, 20 days per month (i.e., during market hours) will have a
request frequency of 567,000 refreshes/month = 60 seconds/minute
x 60 minutes/hour x 8 hours/day x 20 days/month. This is a good
back-of-the envelope example, because even if you are only
caching a single symbol, it is clear that a 600,000 hit
subscription plan is the lowest possible solution. In the case of
end-of-day updates to a historical database, the frequency is
simply the number of working days per month, usually 20 or 30
depending on the application. And, ad-hoc downloads can be as
little as one per month for reference data or fixed rates that do
not change very often.
Simplify the Data
Block
After selecting your trigger event
scenario, the next step is to choose the simplest possible data
block to make the hit calculation as easy as possible. In
general, choosing a data block that equals one hit vastly
simplifies your hit calculation, because it equates each hit with
the natural data unit of your application, e.g., a
symbol.
Single Hit Requests
Xignite pricing is very uniform across all Web services,
so most hit calculations can be done with a simple, natural data
block that equals one hit per request. For example, most quote,
price and rate Web service operations cost one hit for each
symbol across all types of asset classes (i.e., stock, currency
pair, metal, interest rate, etc.) When the hit calculation uses a
single symbol data block, the formula above reduces to the
following.
Monthly Hits =
/> 1 Hit per Symbol x # Symbols per Request x Request
Frequency x 1 month
or
simply
Monthly Hits = # Symbols per Request
x # Requests per Month
For example,
the hits required by a stock index widget would equal the number
of indices in the widget (symbols per request) times the number
of page views per month (request frequency).
src="http://xignite.web-services-blog.com/media/xignite-web-services-usage-single-hit.png"
alt="xignite web services usage single hit" />
Choosing a data block that equals one hit
vastly simplifies your hit calculation, because it equates each
hit with the natural data unit of your application, e.g., a
symbol.
Even if your particular
implementation relies on operations that take multiple symbol
inputs, such as the href="http://www.xignite.com/xquotes.asmx?op=GetQuotes"
target="_blank">GetQuotes operation of the href="http://www.xignite.com/xquotes.asmx"
target="_blank">XigniteQuotes Web service, you can
still use the single symbol, single hit request data block as the
basis for your hit estimate, because the pricing is uniformly one
hit per symbol and will be the same whether you request multiple
symbols at a time using the GetQuotes operation or one symbol at
a time using the href="http://www.xignite.com/xquotes.asmx?op=GetQuotes"
target="_blank">GetSingleQuote operation.
Varying and Random Hits Requests
Occasionally, your application may make use of
operations that do not have a fixed, uniform hit count per input
symbol. For example, you may use an operation like the href="http://www.xignite.com/xrates.asmx?op=GetRateFamily"
target="_blank">GetRateFamily from the href="http://www.xignite.com/xrates.asmx"
target="_blank">XigniteRates Web service where the
number of hits per request varies by the number of interest rates
in the particular rate family requested, e.g., Libors, Euro
Dollars, etc. Or, you may use an operation like href="http://www.xignite.com/xoptions.asmx?op=GetEquityOptionChain"
target="_blank">GetEquityOptionChain of the href="http://www.xignite.com/xoptions.asmx"
target="_blank">XigniteOptions Web service where the
hits per request varies randomly for each input symbol depending
on the number of options contracts in the option chain. In these
cases, you can dramatically simplify the hit calculation by using
the average hits per symbol from the average output data block
(e.g., average # of interest rates per rate family, average # of
option contracts per chain, etc.) When the hit calculation uses
the hits for the average output data block per symbol, the
formula above reduces to the following.
Monthly Hits = Average Hits per Symbol x #
Symbols per Request x Request Frequency x 1
month
For example, the hits
required by an option price research application would equal the
average number of contracts per option chain (average hits per
symbol) times the number of options reviewed each day (request
frequency), times the number of working days in 1
month.
Take a Swag the Request
Population
It's best to approach the request
population calculation with the end in mind. If you've identified
your natural trigger event and chosen the simplest possible data
block, then the request population is always measured in [data
blocks per trigger event], e.g., symbols per page view. The goal
then is to come up with this number.
Request Population = # of Data Blocks
Requested per Trigger Event
If you
use the single hit data block described above (e.g., symbol,
currency pair, fundamental, etc.), then the calculation reduces
to the following.
Request Population
= # of Symbols (currency pairs, fundamentals, etc.) per Trigger
Event
However, the request
population is very dependent on the application and there are
very few rules-of-thumb to simplify the calculation further.
Often the request population must be built from smaller pieces,
e.g., the number of symbols in each widget times the number of
widgets per page, etc. And, sometimes these pieces are random,
e.g., the number of symbols in a user's portfolio widget. That's
the bad news. The good news is that you only need to get the
request population estimate within a factor of 10X to choose the
right subscription plan! So, it's not a problem to gloss over the
details.
Fixed Request Population
Many usage scenarios entail a fixed, unchanging request
population for each trigger event. For example, a daily
end-of-day update to a stock fundamentals database will have a
fixed request population equal to the number of stocks in the
database times the number of fundamentals tracked for each stock.
Whoa, not so fast. How did you get that? You get it by beginning
with the end in mind. The href="http://www.xignite.com/xfundamentals.asmx"
target="_blank">XigniteFundamentals Web service
counts hits uniformly across operations as href="http://www.xignite.com/xfundamentals.asmx?op=GetFundamentals"
target="_blank">one hit for each fundamental data
point returned. Therefore, the single hit data block is one
fundamental. Applying the formula above gives the data population
as the number of fundamentals per daily update. This in turn is
equal to the number of stocks in the database times the number of
fundamentals tracked for each stock.
Random
Request Population
Undoubtedly, the most
difficult hit calculations occur as a result of a random request
population. How can you calculate the number of hits for an EOD
portfolio accounting system when the number of stocks in a user's
portfolio varies randomly by user from one to a hundred? It's
easy! Just take an average and treat it like a fixed request
population, because you only need to get it right within a 10X
factor to choose the right Xignite subscription plan. If the
average user has twenty stocks in her portfolio, then the data
population is 20 symbols per daily update.
Share and
title="email" alt="email" class="sociable-hovers" />
title="Twitter" alt="Twitter" class="sociable-hovers"
title="Facebook" alt="Facebook" class="sociable-hovers"
title="LinkedIn" alt="LinkedIn" class="sociable-hovers"
title="Digg">
src="http://xignite.web-services-blog.com/wp-content/plugins/sociable/images/digg.png"
title="Digg" alt="Digg" class="sociable-hovers" />
title="StumbleUpon" alt="StumbleUpon" class="sociable-hovers"
title="Technorati" alt="Technorati" class="sociable-hovers"
title="Suggest to Techmeme via Twitter" alt="Suggest to Techmeme
via Twitter" class="sociable-hovers" /> rel="nofollow" target="_blank"
href="http://delicious.com/post?url=http%3A%2F%2Fxignite.web-services-blog.com%2F2011%2F02%2Fsimplifying-hit-calculations-for-xignite-web-services%2F&title=Simplifying%20Hit%20Calculations%20for%20Xignite%20Web%20Services¬es=This%20post%20guides%20you%20through%20the%20process%20of%20simplifying%20the%20hit%20calculation%20for%20Xignite%20Web%20services%20to%20something%20you%20can%20do%20on%20the%20back%20of%20an%20envelope."
title="del.icio.us">
src="http://xignite.web-services-blog.com/wp-content/plugins/sociable/images/delicious.png"
title="del.icio.us" alt="del.icio.us" class="sociable-hovers"
title="Google Bookmarks" alt="Google Bookmarks"
title="Live">
src="http://xignite.web-services-blog.com/wp-content/plugins/sociable/images/live.png"
title="Live" alt="Live" class="sociable-hovers" />
title="FriendFeed" alt="FriendFeed" class="sociable-hovers"
src="http://feeds.feedburner.com/~r/Xignite/~4/s0mL2_YvJrQ"
height="1" width="1" />
