Statistical Observations in Gotham
After a product goes live and starts to sell and ship out, unless the product is connected to the Internet, it's hard to estimate engaged usage, but it's nearly impossible to estimate the competitor's total customer population and their engaged usage. The following little trick is something that only works in a dense enough population area like New York or another major metro.
When I was designing navigation devices for in-car use, I spent more time than I will admit to looking inside parked cars. I was looking for evidence of a navigation system such as an in-dash unit or a telltale circular suction-cup mark on the windshield itself. But the holy grail is seeing an actual device out in the wild. When it comes to an early market, with neither your own or your competitors' products ranking highly on adoption, this is sometimes the only way to put the finger to the wind.
The key to this is in a high population high density area, at most hours of the day, and especially rush hour, places like Times Square, ball games, major city events and so on are packed with large amounts of people and this situation lends itself towards statistical observation. For example, by simply noting how many of your own product ships out to the city, it's possible to get a "number of sightings per day" and correlate that to the known total population of devices shipped. And then by noting the number of sightings per day of the competitors' products, it's possible to come up with a pretty reasonable comparative estimate of what you're up against. This isn't rocket science, but it's a quick measure of the iceberg that lies beneath the surface.
Using this technique, it's also possible to observe inflection points in a product's adoption cycle--the iPod to iPhone transition for example, or the steady progression of Amazon's Kindle, can be easily considered just by walking through any subway car.
Also important, when possible, observe the usage scenarios, and the demographics of the users of the devices. Even without shipping a single device, this sort of observational study would yield plenty of information on the types of users out there in the wild. This could lead to a decent user profile or persona for later use.
One of the startups I've mentored came to me with an idea for a new physical product. While I'm no clairvoyant, I quickly identified a proxy product of very similar characteristics and asked the budding entrepreneur to spend days roving the streets and subways of New York City doing nothing but counting. And what occurred over the course of a few weeks was one of the most spectacular pivots. The startup is currently in stealth mode but the product undergone changes that has taken it from a niche product to quite possibly a globally relevant platform that is ready to scale.
So keep your Evernote ready or a Moleskine in your back pocket to do quick tallies whenever possible!