Estimating Millions of Dynamic Timing Patterns in Real-Time

(Title was originally Updating Timing Profiles for Millions of Customers in Real-Time) In some business applications, the transaction behavior of each customer is summarized separately and the summary or customer profile is used to track customer behavior. A customer's profile for buying behavior, for example, may contain information on the likely place of purchase, value of gods purchased, type of goods purchased, and hour-of-day and day-of-week purchases. A customer's profile may be updated whenever the customer makes a transaction, and because of storage limitations, the updating may be able to use only the new transaction and the summarized information in the customer's current profile. Standard sequential updating schemes, such as exponentially weighted moving averaging, can then be used to update a characteristic that is observed at random, such as amount of the purchase. But timing variables like day-of-week are not observed at random, and standard sequential estimates of their distributions can be badly biased. This paper derives a fast, space-efficient sequential estimator for timing distributions that is based on a Poisson model with a periodic, piecewise constant rate that may evolve over time. The sequential estimator is a variant of an exponentially weighted moving average. It approximates the posterior mean under a dynamic Poisson timing model and has good asymptotic properties. Simulations show that it also has good finite sample properties.