Researchers track the trackers through 20 years of the archived web

Everywhere you go on the web, trackers are working to reconstruct your every move. But who tracks the trackers? That’s just what researchers at the University of Washington are doing, with purpose-built tools and liberal use of Internet Archive’s Wayback Machine.

The Tracking Excavator project is an effort to quantify a trend we all have more or less accepted as axiomatic: as the web has grown, third-party trackers, from advertisers to analytics, have grown with it. It’s one thing to accept something as true, however, and another to show it and analyze it.

“Until now, we didn’t have the tools to understand how these approaches have changed since the earliest days of the web,” said Tadayoshi Kohno, from UW’s Security and Privacy Laboratory, in a news release. “Now we can see how the quantity and variety of trackers has grown, and how some approaches have fallen out of favor while others are on the rise.”

Not that it was easy. The team, led by grad students Adam Lerner and Anna Kornfeld Simpson, was working with a highly incomplete data set, much like the fossil record. Their primary source was the Internet Archive and its Wayback Machine. But that tool, for all its usefulness, was designed for savingcontent, like blog posts — not metadata or offsite code and scripts.

“We had to develop techniques to extract tracking information from the archive,” said Kornfel Simpson. “For example, we collected tracking cookies from archived HTTP headers and Javascript and then simulated the browser’s cookie storage behaviors to detect tracking behavior.”

No shortcuts there — it took the team a year to sort through 20 years of records, starting in 1996. But the resulting tools and dataset are original and valuable.

The study was presented at the USENIX conference in Austin, and represents a sort of first pass at the data; deeper analysis is forthcoming. Among the facts they did discover is that the amount of third-party tracking happening on the average website has increased by a factor of four. That is to say that, in 1996, there was maybe one tracker on most websites, and now there are generally at least four.

Individual trackers cover a larger part of the web, as well: no longer will an advertiser or analytics company be found only in a thin cross-section of sites. Instead, the biggest ones are now present on 20 to 30 percent of all sites tested. The trackers themselves have become more complex, watching and correlating many types of behaviors across many sites.

These results do sound intuitively true, but the dataset collected by the team makes it possible not just to confirm them, but to graph them over time and link them to other developments with changes observed in it. Did ad-blockers make a dent or spur growth? Did improving web standards change the way trackers worked? What about changes in consumer spending and the economy?

The “Tracking Excavator” the researchers built to automate some of the tracker-tracking process isn’t available for download yet, but they plan to release it soon. In the meantime, you can browse through the data they collected and perform your own analysis, if you dare.

Walmart is buying Jet.com for $3 billion

Walmart Stores is buying Jet.com in a deal worth $3 billion dollars according to a source with direct knowledge of the deal, confirming reports that have been pouring in about the bid for Jet.com all week.

According to our source the signatures for the deal were dry on Friday and will be announced as early as Monday morning — echoing what was reported in both Bloomberg and Recode.

The deal for Jet.com comes as big brand companies and retailers are trying to shore up their defenses against an all-out assault on their business from Amazon.com.

From its launch in 2015 Jet.com set itself up to be an Amazon killer. The company was founded by  Marc Lore, who sold his previous business, Quidsi, to Amazon for $545 million.

The operator of sites Diapers.com, Beauty.com, and Soap.com, Quidsi was part of an Amazon buying spree that included the retailer Zappos.com and Woot and established the online retailer as a huge competitor in sales of both clothing and consumer packaged goods.

Now, brands and the biggest box stores are fighting back with their own billion dollar acquisitions.

One of the reasons that Unilever bought Dollar Shave Club in last month’s big dollar consumer deal was to compete with the looming threat of Amazon… and Walmart faces similar pressures.

The deal is a nice payday for Jet.com investors who had committed more than $800 million in financing into the company. Investors like Accel, NEA, General Catalyst Partners, Norwest, Goldman Sachs, and others all stand to gain substantially from the Walmart acquisition.

But the biggest winner of all may be Lore himself. With a 25% stake in the company Lore stands to make as much as $750 million from the sale. As well as take the helm of the ecommerce site of commercial retail’s largest player. A company that raked in over $15 billion in profit last year. Such a large payout sends a strong message from Walmart about Lore’s expected value-add to the brand.

Since its launch, the company had grown furiously, but its growth was coming at a steep price. According to the report in Recode the company was spending $20 million to $25 million on marketing to fund its growth.

Indeed, Jet.com’s CEO even noted at the time that the fundraising environment for his company had become “tough”. And the company’s own public statements indicated that it wouldn’t be reaching profitability until at least 2020.

Even as Walmart latches on to Jet — and its superstar e-commerce CEO — as a potential savior for its online sales woes, it may be looking in the wrong direction.

As our own Sarah Perez reported last year, Jet has not made much of a dent in Amazon or eBay sales:

… if Jet’s strategy is to lure customers away from Amazon, that, so far, has not happened… they’re not yet seeing any cannibalization of Amazon or eBay sales at this time. In other words, people are buying on Jet, but their purchase rate remains consistent on Amazon and eBay. That could mean that Jet is succeeding instead in gaining customers who would have otherwise bought products via other discount marketplaces, like Costco or Sam’s Club, for instance.

Notably, Jet.com started with membership fees similar to Amazon Prime. Prime has been a knock-out for Amazon with some predicting the company has over 54 million Prime members in the US. Despite Amazon’s proven success monetizing e-commerce with a membership strategy, Jet.com ditched memberships and instead focused cutting prices for all.

While a savings strategy would add depth to the brand image of most other retailers, Walmart is already well known for its mantra of “Always Low Prices.” A Jet.com acquisition doesn’t inspire confidence that Walmart will suddenly become more well known for savings. And even if it did, low prices don’t seem to be enough to fix Walmart’s e-commerce woes.

Despite having a brand synonymous with savings, Walmart has seen five straight quarters of declining online sales growth.  The millennial user-base of Jet.com has the potential to add a spark to Walmart’s current brand image, but it’s unclear the degree to which such an effort differs from prior investments made by Walmart Labs in big data, open source, and cloud services.

If the strategy is just to throw punches at Amazon, Walmart has targeted the company’s users with new grocery services, faster delivery, and Walmart Pay. The company even wants to use drones inside its warehouses. Alternatively, for the Jet.com acquisition price, Walmart could have broken from the path of its rivals and bought five million Oculus Rift headsets to put users in a virtual reality capitalist paradise.

Medium nabs Embedly to add to its list of publisher tools

Medium announced today that it has acquired Embed.ly to support publishers with backend APIs for embedding content.

Embed.ly supports writers by providing analytics on content and customized recommendations and promotions. Their APIs are currently used by an all-star list of publishers including The New York Times, NPR, and The Atlantic. Companies like Reddit and Airbnb also use the service.

At this point, the company is servicing 500 million API requests per month. The team will continue to operate independently of Medium and is currently pursuing innovations in embedding native video.

The analytics features of Embed.ly let publishers see who is clicking on what, and the number of plays and minutes watched for video.

“We want publishers to understand why content is doing well,” said Kate Mason, head of communications for Medium.

Medium is doubling down on publisher tools. Earlier this year the company acquired Superfeedr, a company that produces APIs to help produce feeds quickly and push them to the right places.

As part of its services, Medium rolled out a beta monetization program allowing users to promote stories as an advertising unit. Additionally, the company has a membership program that enables content creators to lock some content behind a paywall.

At a greater level, Embed.ly has consistently strived to standardize embedded content to make life easier for both content creators and content consumers.

“The problem with embeded content is that there are no good standards,” said Sean Creeley, co-founder of Embed.ly. “We want what appears amazing on Medium to look great on WordPress VIP and other platforms. Medium gives us the ability and platform to create really great open standards around embeds.”

To date, the company has raised $1.02 million in both equity and debt financing from Y Combinator, SV Angel, Lowercase Capital, and others. According to PitchBook, the company’s most recent valuation was $3.9 million in 2011.