Technology and Products

Eddystone and Beyond

July 29, 2015
Martin Sahlen
Platform Engineer

Google launched Eddystone a couple of weeks ago. This piece is about Eddystone seen from a technological perspective, and how it could impact the (still very early) proximity industry - for the PSPs, for brands & retailers as well as for end users. Firstly, a credit is due to the Radius dev blog as well as Doug Thompson on the Beekn blogRichard Graves from Bkon on the Bkon blog and many more - there are already a plethora of insightful pieces on Eddystone. I hope by this post to add to that, standing on the shoulders of the giants, as a wise man once said.

This piece is divided into two parts, one on the technical side and one on the more market-oriented side, with some degree of overlap between them - but repetition is always good. For the impatient readers, the main conclusions are served here:

  • Eddystone will increase the market size and adoption by bringing a consistent experience to Android and iOS for both developers, PSPs and the end users
  • Google is a taking a position by offering a companion suite of products that taps into the current PSP product offering. However, unclarity on data ownership and privacy may make brands and retailers reluctant to jump on board. The role of PSPs as independent and transparent regarding data ownership and privacy is a strong advantage as well as a more mature product offering
  • The physical web can increase the market size dramatically by lowering the barrier to entry and reducing friction for both developers, brands and retailers and end users. The granularity and quality of the data in the physical web does have a different nature than when using native apps; for indoor navigation and advanced indoor analytics such as dwell time and similar, native apps still seems the way to go. However, using logins, cookies and usual web tracking mechanisms you are still able to get a fair amount of data from the physical web
  • The physical web and the Chrome’s ability to open URLs from beacons will open many new possibilities, but not necessarily only from a pure commercial aspect. The democratic nature of the physical web will likely yield crowdsourced projects where people can collaborate on mapping up physical spaces (for instance with Wikipedia articles)
  • The physical web may act as an introduction or preview for potential clients into proximity as a marketing tool, and later funnel them into more feature rich app environments, again strengthening the position of mature full stack PSPs
  • Proximity is here to stay

Part 1 - Technology overview and impact

Eddystone - what is it

Eddystone is not so much a protocol rather than a “suite” of very different protocols, or more specifically frames that is emitted via BLE. Currently, the official frames supported is UID, URL and TLM (telemetry). The spoken goal of Eddystone is a “Flexible architecture permitting development of new frame types”, so we should assume future development here as the market matures and the maintainers (i.e. Google) get more feedback from the market and the users. This means that we may see many new types of frames for different purposes as Eddystone matures, but there will also be a balance between having a confusing number of frames as well as the battery life implications on both devices and beacons to handle this and advertising / consuming the frames at regular intervals.

Eddystone-UID

Eddystone-UID works semantically almost the same as iBeacons, where the UID consists of a namespace ID (first 10 bytes) that resembles a region in iBeacon, and an instance ID (6 bytes), which maps onto the major / minor we see in iBeacons. This allows for filtering the UID based on namespace (do you see the similarity with region monitoring on iOS?) and other optimizations. Whether there are any restrictions on the number of namespaces here is not stated. There might be no hard limits (as in the ~20 regions on iOS), but battery concerns will probably prohibit too many namespaces in practice. Hopefully, more information will be available on this matter.

Eddystone-URL

Eddystone-URL contains in it essence an URL that follows some convention regarding length and protocol (currently only http / https and only a subset of domains, and the URL must also be shortened) that is broadcasted and can be read by all receivers regardless of the Eddystone-UID (hence the advantage of using dedicated frames for different purposes).

As Eddystone-URL-beacons are not “consumed” with native apps, but rather with a web browser (for now Google Chrome), there also some other aspects and limitations that come in to play. For a brand to use this, there are quite different mechanics than with a native application that uses Eddystone-UID or iBeacons. It will require the brand to create a web application that hosts content for all the URLs. After an Eddystone-URL is encountered, the browser shows the page just as if you would have entered by typing an address or clicking a link.

However, we know nothing of how the monitoring and scanning for Eddystone-URL is implemented in the browser application itself (which is a native app), and if Google is tracking these events for aggregating user data. As Eddystone-URL-beacons are dealing with web pages, in order to get useful data, PSPs would need to have REST endpoints to gather browser data on the users, that would work similar as Google analytics or any other vendor that tracks a user’s browser history and behaviour online. By using regular logins, user data can also be tracked.

Technically, if (when?) Eddystone-URL catches on, PSPs would need to update and / or create APIs and data models that also encompass information and semantics for web / cookie tracking to support data originating from these beacons. It will essentially mean that PSPs must extend the taxonomy of entities that generate events to include web pages. I.e. a beacon can generate a ‘dwell’ event, while a web page can generate a ‘pageview’ event. It would also mean that matching users across devices will be more difficult, as will creating consistent and valuable analytics on a granular user-centric level.

Eddystone-TLM

Eddystone-TLM is data such as Temperature, Battery level and other sensor readings. This may open for broader adoption within industrial applications where sensor data is critical to operations such as oil, different kinds manufacturing processes, freight and more.

Hardware implications, battery life etc

Supporting Eddystone requires no more than a firmware upgrade to a beacon (remember that it relies on BLE), which may already transmit many frames such as iBeacon, AltBeacon, perhaps URIbeacon as well as proprietary maintenance frames that are used for fleet management etc (although this is now formalized in the Eddystone TLM, most major vendors are doing this in their own formats). The interesting part is that there is a convergence towards very advanced beacons - the major vendors now ship beacons that support all existing standards, meaning that the beacons must emit many types of frames. It will be interesting to see how this affects battery life on the beacon (and if this really poses a problem as more and more beacons go on mains / USB power) as frames should be advertised at a rather high rate to give users the snappiness and response they expect.

Concerning battery life, it will also be interesting to see how Eddystone affects battery life on iOS, as it is not a deeply integrated and optimized system library such as iBeacon. According to Radius Blog, “Eddystone actually uses a fourth frame, which is a standard iBeacon frame. The primary purpose of this frame is so Eddystone can leverage the iBeacon standard's special ability to wake up iOS apps in the background, at which time they can start consuming the three frames above.” Then, Eddystone uses the regular BLE-library for iOS and Google also have a reference implementation on github to show how this works. Eddystone is compatible with Android, iOS, or any platform that supports BLE beacons. However, the target is iOS and Android for the time being. Eddystone is currently available on GitHub under the open-source Apache v2.0 license.

Complimentary products, services and initiatives

Google seems to be actively taking a position within the proximity industry with the launch of Eddystone, and it becomes even clearer when we see the companion suite of products that was launched in the wake of Eddystone. Although clearly immature and somewhat lacking in functionality and documentation, it may show where they are going. These are the main initiatives from Google’s side in building their own ecosystem (some, such as Places and Nearby have been around for some time):

There is a lot of documentation and descriptions on these links, so we will not go into great detail here. However, the main learning here is that Google has now launched products that allow anyone to deploy beacons and manage them in the Google cloud. This includes tasks such as fleet management and monitoring, registering beacons, provisioning beacons, enabling, disabling them as well as mapping assets to the beacons, all using the Proximity beacon API. They have code samples that show how you can listen for beacons directly using iOS or Android in the nitty-gritty way or you can use the Nearby API (that they obviously recommend) that automatically looks for nearby devices and beacons and get any information / assets attached using the Proximity beacon API under the hood - doing the heavy lifting. There are also integrations with the Places API for doing things with Google maps as well, completing the full service offering and combining proximity and location data in Google’s ecosystem.

Obviously, all this can be taken with a grain of salt, as the APIs and docs are still somewhat wanting for clarity and information. The current PSP offering is vastly ahead in terms of analytics and feature sets, but one should expect this to rapidly evolve as (or if) Google gains momentum and feedback on Eddystone and their surrounding initiatives.

The plot thickens - Service worker

Google is pushing on w3c to implement the service worker as a browser standard. It is currently a draft document, on which two out of three authors are Google employees - indicating that they are a driver in getting this into our (mobile) browsers.

From the draft:

The specification describes a method that enables applications to take advantage of persistent background processing, including hooks to enable bootstrapping of web applications while offline. The core of this system is an event-driven Web Worker, which responds to events dispatched from documents and other sources. A system for managing installation, versions, and upgrades is provided. The service worker is a generic entry point for event-driven background processing in the Web Platform that is extensible by other specifications.

Service workers are generic, event-driven, time-limited script contexts that run at an origin. These properties make them natural endpoints for a range of runtime services that may outlive the context of a particular document, e.g. handling push notifications, background data synchronization, responding to resource requests from other origins, or receiving centralized updates to expensive-to-calculate data (e.g., geolocation or gyroscope).

This indicates that at a point in time, we may receive push notifications from web pages within browsers, as we receive it from native apps today. There are already browser extensions and web pages that do this on desktop browsers. This maps very closely up with Google’s work on the physical web / chrome, which currently lacks many essential pieces to be at a level that competes with native applications.

Part 2 - Significance and impact for players and incumbents in the beacon space

Eddystone-UID

Eddystone-UID in itself works on most levels just as regular iBeacons and can also be used as such, meaning that a PSP can create they own beacons and SDKs using this without little or no changes. It is very important to emphasize that Eddystone is an open source project anyone is free to use regardless of Google and anyone can maintain their own beacons, SDKs and applications leveraging Eddystone without Google having any part in this. An end user on a native application would never know the difference between Eddystone-UID beacons and iBeacons - they fulfill the same use case (given that Eddystone will show to not affect battery life harder than iBeacon). There is nothing requiring any PSP (or anyone else for that matter) to have a link, agreement or affiliation to Google in any way to use Eddystone.

However, what Eddystone-UID does, is to provide a coherent SDK / API that works on Android and iOS seamlessly (in time, we hope) which minimizes burden on the PSPs to support all different protocols and giving developers in general more time to focus on delivering value and create cool use cases. It also opens up the beacon market in a scalable (battery life, best practices, etc) manner for all 1 billion++ Android devices out there, which have been a bumpy road so far - limiting user adoption.

To be very clear, the Eddystone-UID is more or less only positive, as it increases the market and pie for everyone. Obviously, who will get their cut is another discussion, but it is very difficult to label any development that increases the market size a negative one - even if the development is Google-branded.

Eddystone-URL

Eddystone-URL is a totally different beast than Eddystone-UID and iBeacon. The use case is a bit different, and the nature (currently at least) is more of a pull than push i.e. users seeking information on nearby objects rather than being pinged.

What is interesting about that is that the Eddystone-URL is universal and anyone can make an application that consumes the Eddystone-URL frame and shows the content a browser view. Eddystone-URL opens up a proximity market that are orders of magnitude larger than the current one by bypassing the need for a native app. Both because the barrier of entry is lower both for developers, brands and retailers. Web pages are easier and cheaper to make than apps, as well as being more accessible to users instantly without forcing them to download and application.

The democratic nature also opens up for crowdsourcing of content - imagine Wikipedia being mapped up by Eddystone-URL, as anyone can buy and deploy these beacons for others to consume. From the commercial point there are obviously many use cases here, but an important demarcation line is that the ability to collect proximity data is low, as well as the granularity of the data collected and the analytics that can be inferred from it. Which, for some use cases, might not matter. Especially, in cases where privacy is not a factor and the requirements can accomodate using Eddystone-URL it hits a bulls eye. It’s easy to deploy and cost effective.

In many cases, Eddystone-URL can be a good thing, because it might allow more brands and retailers to play around with this technology and get comfortable with it - before resorting to a PSP to actually delivering an end-to-end solution that gives them more granular data and advanced use cases. Eddystone-URL can also work in tandem with Eddystone-UID by showing a link to download an app, lessening the friction experienced by the user.

Complimentary products / services and initiatives from Google

As discussed in Eddystone-UID section above, in order to use the features of Eddystone, there are no strings attached for anyone wishing to do so. However, Google as conveniently enough also launched a “complete” set of features for managing beacons as described in detail previously in this document. This could be viewed as threatening to existing PSPs, as it commoditizes their product offering - and considering the scale and size of Google they will be able to put large efforts into developing this further (the current level and sophistication is far behind most existing PSPs, but Google is Google).

It will be naive at best to believe that Google are not aiming to utilize proximity data in their own channels. This data is gold for Google’s advertising operations and user profiles as even one more source in their array of data inputs. The Proximity Beacon API and its siblings may thus be no different than Google’s range of free products where users “pay” with their data. The only difference now is that “users” of this product are in fact companies that are paying with their users’ data to a third party - quite a different scenario indeed.

Further relating this to the fact that publishers, brands and media actors are increasingly aware of the substantial value of their user data (CRM, Transactions, Web, Apps, Proximity++), we should expect that they will be reluctant to jump on the Google train right away. As data ownership and privacy is a hot topic both on a personal and corporate level today, this makes a lot of sense. For instance, questions such as who buys your data, what data is collected and how / where is it used are questions that many brands and retailers cannot just leave to chance or the mercy of Google.

Using Eddystone-UID in native apps will not have any impact unless Google’s APIs are also used in conjunction. The use of Eddystone-URL might be prohibited on a commercial level by unclarity of how data is logged and used by Google in the browser app (and on the web for that matter; Google has full control as they are implementing the browser application) before sending a user to the URL. The potential seems much greater in areas such as cities, parks, museums etc. It also (currently) seems difficult to create user experiences on the same level as with native apps and “regular” beacons, but this might change with the introduction of service workers, enabling push notifications from web pages.

The role of PSPs as transactional service providers that charge a Saas fee can very much be seen as a strength; they are actors that are “neutral” and funnels data into whatever channels the brands and retailers wish to use and keeping them in complete control over data, access, privacy, monetization and channels in a transparent way while charging in cash, not by extracting data (Google might very well be one of these channels that brands wish to use, but it shouldn't be the only one).

Further, even though some will jump on the Google train and their suite of proximity products, we should keep in mind that Google has about 30% of the advertising market. This is a large portion, but there is 70% remaining. Assuming that there is low control over the data collected and how it used, there is little possibility to utilize the collected data outside Google’s channels. However, if you have no media channels, DMP or an agency, you can still do a lot within Googles ad universe - if you are willing to accept low transparency and control. Perhaps Google’s bet is that many smaller companies with limited budgets will do this. It is however important to remember that EU within a few years introduces new legislation on privacy that will really hurt companies who are sloppy (the fines companies are facing will not be fixed, but rather a percentage of the revenue).

Considering the privacy and data ownership and data control issues discussed above and a 30% ad market reach through Google’s channels, there seems to still be a substantial space for PSPs and other actors to navigate.

Google’s proximity products have some similarities to a Trojan horse, but as there is increasing focus on the problematic areas there will be cautious to jump fully on board the Google train. In this regard, the position of independent companies is still valuable - giving brands and retailers full control of their data, paying a premium for this in hard cash - not by compromising their data or their customer’s privacy. In these times, that seems like a manageable and justifiable tradeoff.

Further readings

These readings are going further into some of the topics in this piece, this piece is in many ways a synthesis of these. Especially, credits goes out to the Radius Blog and Doug Thompson in the Beekn Blog. Please scroll down and read them in their entirety; some also contain interesting and pedagogical videos that are very helpful in increasing the understanding - especially from the tech side of this. Most are pretty short and neat and to the point. Some articles are more on the businessy side, while some are very techy. Some, such as PHY.net is an interesting initiative where Bkon has made a “layer” between between the physical web and the URL-enabled beacons, essentially a CMS for Physical web beacons.

Sources