Introduction


Datasnap.io ingests events generated by apps interacting with beacons and running campaigns triggered by those beacons. There’s loads of info about how beacons work on the inter-web as well as a ton of beacon providers to choose from. Some beacon providers include Estimote, kontakt.io, Gimbal and many more…


The datasnap.io platform doesn’t care what kind of beacon you use to gather proximity data! You can send us data from any platform and we’ll suck it into the system and run our algos on it to understand metrics like dwell time and location engagement. We’ll create beautiful reports for you on campaign lift, user traffic in a venue and most traveled paths in a venue. You’ll also have complete access to the processed data via our Analytics API’s or your own redshift cluster.


The following document provides an overview of integrating with the system. Our complete technical documentation is hosted on Apiary. This is the source of truth for all of our technical API documentation. You can use Apiary to send test events and see how the system would work.

Contact us if you need any assistance at all support@datasnap.io


 


Beacon Installation


Beacons

Beacons send out periodic Bluetooth low energy signals that mobile devices detect. Applications that interact with beacons can provide context about proximity and user patterns. Using this information, the app can send users notifications or other interactions. For example, Virgin Airlines places beacons near the security gate, so travelers with the Virgin app installed get their boarding passes loaded automatically when they are near the security gate. Users must opt in before the app interacts with beacons. Beacons cannot send push notifications or proximity information without consent.

Beacon Setup

Think about your core goals for these beacons:

  • Are they just to monitor traffic flow?
  • Are they triggering a push notification based on user location?
  • Are you trying to drive people into the store using proximity-based messaging?

Signal

  • Beacons are best placed above people’s heads and away from anything that might block the signal. 10 feet above the ground appears to be optimal.

  • Place beacons where they will be visible to people’s mobile devices. Beacon signals generally range from inches to about 300 feet. Signals can be affected by physical barriers, such as walls, people, or structures. Placing beacons in a clear, unobstructed area will improve results.

  • Some beacon vendor signals are more consistent than others.

Proximity and Places

  • If your app needs to know how close users are, place as many beacons in the targeted area as possible. Ideally, a person should be sighted by 3 or 4 beacons at all times within the targeted area.

  • Likewise, if your app needs to determine when someone passes by, place a high number of beacons in the area to better detect how close people are. This is useful for measuring wait times, queue times, etc.

  • Alternatively, to simply determine whether a person is present, set up a zone or place to detect when users are in the area. Group a number of beacons together to to define a place or zone. Accuracy is not as important with places, and this grouping effectively determines when a user is near or far from that place.

RSSI Triangulation

  • Measuring distance from signal strength (RSSI) is a tough task because RSSI is easily affected by its environment. RSSI signals can be blocked by walls, people, and structures. It may also be reflected by various objects, resulting in a very noisy signal. Beacon and mobile device orientation also affect RSSI, making it difficult to infer accurate distance information without deep knowledge of the physical location and the beacon.

  • A possible solution is “fingerprinting,” whereby a table of expected RSSI values corresponds to various places of interests. Compare user’s readings to this table to improve triangulation.

Beacon Monitoring and Maintenance

  • Most beacon providers give you a way to monitor the beacon battery.

    • Generally if the battery status is yellow, then replace the battery as it tends to fade quickly thereafter.
  • If possible, mark beacons with Latitude and Longitude coordinates.

    • Beacons can be plotted on a map using Lat/Long coordinates. This is useful for displaying population density data on the map.

    • This usually works best in large spaces like a festival or stadium.

  • Group beacons by tagging them: assign them to a place or tag them with labels.

    • Beacon groups are better at tracking movement and pathing than singular beacons.

    • Tags identify beacons zones, useful for studying visitor density and pathing to other zones.

Beacon Manufacturers

Manufacturer Details Price Spec
Gimbal
  • Best hardware
  • Rotating UUID security model
  • APIs/SDKs have decent documentation
  • Least expensive
Small: $15 for 3 beacons
Large: $20
Gimbal Spec
Estimote
  • Sticker hardware available
  • Rotating UUID security model
  • Good SDKs and APIs
  • Quite expensive
Large: $99 for 3 beacons
Stickers: 10 for $99
Estimote Spec
Kontakt.io
  • Under review
$81 for 3 beacons Kontakt.io Spec
Radius Networks
  • Under review
USB: $29
Tag: $39
Radius Networks Spec
 


System Hierarchy


Parent Organization

Parent organization for the organizations and venues. This will be provided for you by a datasnap.io representative. Contact support@datasnap.io to get started.

QUICK TIP - when you set up organizations and venues in the system you MUST send a parent_org_id

Organizations

Datasnap.io segments data and permissions around the notion of organizations. Venues and apps are associated with an organization, so when you send in an event into the system it must always have an associated organization_id.

QUICK TIP - With every event you send into the system you MUST send an organization_id

Venues

Venues represent a physical location where a beacon is placed. You must always send in venue_org_id with an event. Sometimes you might not have a venue, for example you may have put beacons around the streets of London. In this case you can do something like send in ‘Unclassified’ with the event or send in ‘London’ as the venue. Venues are really just another way to segment your data.

QUICK TIP - With every event you send into the system you MUST send a venue_org_id

Advertisers

You can now separate advertisers from organizations. This is typically used in a network model when clients are running campaigns on their apps on behalf of an advertiser who does not own the app or the venue.

QUICK TIP - With every campaign you send into the system you MUST send an advertiser_org_id. If this id is left out of the event the campaign event will default to the organization id.

Projects / Apps

Projects are a way to further segment data. Projects are typically apps (eg: My Cool Android App, My Cool IOS App). A sub-organization may have one or many apps feeding data to it. and project_id along with every event.

QUICK TIP - With every event you send into the system you MUST send a project_id

Places

Places are a further breakdown of locations within a sub-organization. You can use places to group beacons or geofences together. We recommend grouping sets of beacons together into places - like stage left, stage right, front entrance etc. Places are critical to building out and visualizing paths people take from one location to another. For reporting purposes we recommend using as many beacons as you can to group them into places, so people walking around are seen by as many beacons as possible.



Typical Implementation

In a typical setup the entity who owns the app, runs campaigns and owns the venue are all the same organization. In other words you are running campaigns and collecting user location data on behalf of one organization that owns the entire system.



Network Implementation

In a network implementation the key difference is that the entity running campaigns may be a completely separate organization from the owner of the app or the venues. For example coke may run a campaign on a team app at your stadium.




Understanding Permissions

Organizations perform two key functions in the system.

They represent a group to which you can assign users
Users are either administrators or read-only users

They are owners of entities in the system
Entities are apps, venues, beacons, places, geofences and advertisers

Example

The San Francisco Panda Ant’s are playing World Team Tennis at Membley Stadium and Bob’s Cola wants to run a campaign on the Tennis app while people are in the stadium.

“Bob’s Cola” is created in the system as an Advertiser and campaigns are run under “Bob’s Cola” advertiser on the Tennis app. Campaign events are captured and sent to the datasnap.io servers from the tennis app.

The World Looks Like This…

  • There are 2 stadiums owned by “Mega Sports Corp”
  • There are 2 apps. owned by “Xplinga” Publisher
  • There are campaigns being run by the advertiser “Bob’s Beer”




Provisioning User Access


ONLY allow users see to campaigns run by Bob's Beer on the Tennis app


  • The admin creates a group called Marketing Team Group.

  • She places the Tennis app and the Bob’s Beer advertiser into this group.

  • She then takes the marketing team people and places them in the Marketing Team Group.

  • Any users placed in this group will now have access to see campaigns run by the Bob’s Beer advertiser on the Tennis app.



Provision access to Membley Stadium location data collected by the Tennis app


  • The admin decides that Ryan, Kate and Derek should also be allowed to see data generated by beacons in the “Membley Stadium” venue (like heatmaps and user pathing).

  • The admin adds “Membley Stadium” to the “Marketing Team Group”.

  • Now Ryan, Derek and Kate can see dashboards showing venue data from Membley Stadium generated by the Tennis app.


 
 




Sending Data

There are two core API’s for getting the data into the system…

Event API

Use this API to send events into the system from your app. These are events like beacon sightings, beacon entry and exit events and campaign clicks. This API is used to capture all the events you want analyzed in the system. Event API Reference »

Entity API

Use this API to get entity’s into the system. You can think of an entity as a thing that can have meta data attached to it e.g. a beacon. A beacon can have meta data describing it like a description or a lat / long or battery life. Entity API Reference »


Sending Event Data

There are several important id’s in play when reading and writing data from/to the system. These id’s determine how data is segmented and permissioned.

1. organization_id: This can be thought of as a group. Every entity in the system is owned by an organization. When sending in an event the organization_id is ALWAYS the owner of the app.

2. advertiser_org_id: Used when sending campaign events. This is the advertiser that is sending the campaign event

3. venue_org_id: This is the venue that the beacon is located in

4. project_id: This is the application that generated the event

 


Getting Entities into the System

You send in the following entities using the Entity API

  • Beacons
  • Places
  • Geofences
  • Apps
  • Organizations
  • Venues

You send these entities in along with descriptions and any other information like lat / long, battery level, floor and so on.

NOTE: You do not need to send in campaigns via the Entity API.


Datatypes

The typical JSON datatypes are in play when sending data: http://en.wikipedia.org/wiki/JSON#Data_types.2C_syntax_and_example

Property Name Rules

  • Names must be less than 256 characters long.
  • There cannot be any periods (.) in the name.
  • Names cannot be a null value.

Property Value Rules

  • String values must be less than 10,000 characters long.
  • Numeric values must be between -2^63 (-9223372036854775808) and 2^63 - 1 (9223372036854775807) (inclusive).
  • Values in lists must themselves follow the above rules.
  • Values in dictionaries must themselves follow the above rules.


Dates

All dates should be represented as ISO 8601 formatted dates and include the timezone. http://en.wikipedia.org/wiki/ISO_8601

UTC Time

  • 2015-06-21T18:24:18+00:00
  • 2015-06-21T18:24:18Z


PST Time

  • 2015-06-21T18:24:18+08:00
 


Properties

We have the following top level properties that represent the current domain when dealing with proximity data and events. These properties can automatically be updated or they can be updated as part of our Entity API. You must pick one approach to use you cannot do both at the same time. Also note that the entity API will provide for more advanced insertion of data to be used in your reports in the future.

docs.datasnapioentityapi.apiary.io

  • Tags
  • Organization
  • Project
  • Place
  • Global Position
  • Beacon
  • Geofence
  • User
  • Campaign
  • Communication
  • Datasnap


Required Event Fields

Name Description
organization_id organization id assigned to you
advertiser_org_id advertiser id sent along with campaign events
venue_org_id the venue that this event is associated with
project_id Project id within that organization (typically app)
event_type The type of event you are sending - beacon_sighting, geofence_depart etc.
global_distinct_id the unique user id associated with this event




Populating Core Reporting

Beacon Heat-mapping

To get beacon reporting send in a minimum of a sighting event.

Place Pathing

To get place pathing you need to associate beacons with places.

Campaign Reporting

To view campaign reporting send in a communication event with a campaign.

Geofence Reporting

To get geofence reporting send a geofence_exit event.

GPS Reporting

To see GPS reports or user movement send in a GPS event.


Core Information and Events

The core events to send are listed below

  • Beacon Events
  • beacon_sighting
  • beacon_arrive
  • beacon_depart (send dwell_time along with the depart)


  • Campaign Events
  • ds_communication_sent
  • ds_communication_open


  • GPS Events
  • global_position_sighting


  • Geofence Events
  • geofence_depart


  • Device Information
  • With all events send as much device information as possible


  • User Information
    • Events should all send back the same same global_distinct_id to identify a user. This is critical to matching beacon_id to campaign_id to user_id. Also send back to datasnap as many other user ids as possible with the events. By sending as many id’s as possible we can link those to audience information to slice and dice the data further.
 




Uploading Data

Quickly add, update, or delete datasets to the datasnap.io platform via a CSV file. Use curl commands in the terminal to upload data to the datasnap.io server.

Detailed Responses

A 200 response returns when a CSV upload succeeds. To get more information about the upload, use the verbose=true flag in the URL of the HTTP request.

Example cURL URL:

"https://entities.datasnap.io/v1.0/beacon_entity_upload?organization_id=##############&verbose=true"

Example: Verbose Response for Adding and Updating Beacons Successfully

[{"UPDATED":{"venue_org_id":"TEST_venue_1","name":"TEST_name_1","identifier":"TEST_id_1"}} ,{"UPDATED":{"venue_org_id":"TEST_venue_2","name":"TEST_name_2","identifier":"TEST_id_2"}} ,{"CREATED":{"venue_org_id":"TEST_venue_5","name":"TEST_name_5","identifier":"TEST_id_5"}}]

Example: Verbose Response for Deleting Places

One Place the in CSV did not exist.
[{"DELETED":{"venue_org_id":"1tIaudT6OqBKCLhCC4LliE","place_type":"physical_place","name":"The Place Name","id":"place-id"}},{"NOT FOUND":{"venue_org_id":"1tIaudT6OqBKCLhCCaLliE","place_type":"physical_place","name":"Place 2","id":"Place2-id"}}]

Bad Responses

A 400 error response returns if the CSV file is improperly formatted. Some common mistakes are:

  • Values are missing from a row
  • The Action column has a value other than A or D
  • Beacons do not have a venue_org_id column or venue_org_id values are missing
  • Places do not have a venue_org_id column or venue_org_id values are missing


Uploading Beacons

Below are the required columns to upload beacons via CSV. Download a Sample Beacon-Upload CSV to see an example.

Name Description
Action Whether to add or delete a place. Use A for add/update and D for delete
venue_org_id Unique id for the organization
name Optional beacon name, often a venue name
identifier 10-digit string identifying the beacon

cURL Command

Be sure to pass your organization ID into the URL. Use the optional Verbose flag to see a detailed response.

curl -X POST "http://entities.datasnap.io/v1.0/beacon_entity_upload?organization_id=##############" \ -H "Authorization: Basic MkJMTFUxTDJOUjgxOThZMDVZWVFXQ1pDMjpJd0NpSG41T1VXOCtUFFNkVUYW1XXQ1Q0TENvNFRabEc0eXBhZ0FSbmlV" \ -H "Content-type: application/json" \ --data-binary "@beacons2.csv" -v


Uploading Places

Below are the required columns for a place-upload CSV file. Download a Sample Place-Upload CSV to see an example.

Name Description
Action Whether to add or delete a place. Use A for add/update and D for delete
id Place id
name Place name
place_type Value can be physical_place or logical_place
venue_org_id Unique id for the organization

cURL Command

Be sure to pass your organization ID into the URL

curl -X POST "http://entities.datasnap.io/v1.0/place_entity_upload?organization_id=##############" \ -H "Authorization: Basic MkJMTFUxTDJOUjgxOThZMDVZWVFXQ1pDMjpJd0NpSG41T1VXOCtsWUFFNkVUXX1jQ1Q0TENvNFRabEc0eXBhZ0FSbmlV" \ -H "Content-type: application/json" \ --data-binary "@places2.csv" -v


Uploading Place-Beacons

Below are the required columns for a place-beacon-upload CSV file. Download a Sample Place-Beacon-Upload CSV to see an example.

Name Description
Action Whether to add or delete a place. Use A for add/update and D for delete
place_id Place id
beacon_identifier 10-digit string identifying the beacon

cURL Command

Be sure to pass your organization ID into the URL

curl -X POST "http://entities.datasnap.io/v1.0/place_beacon_upload?organization_id=##############" \ -H "Authorization: Basic MkJMTFUxTDJOUjgxOThZMDVZWVFXQ1pDMjpJd0NpSG41T1VXOCtsWUFFNkVUY1jQ1Q0TENvNFRabEc0eXBhZ0FSbmlV" \ -H "Content-type: application/json" \ --data-binary "@place_beacons2.csv" -v




Analytics API

The datasnap.io analytics API can be used to pull data into any system and has a set of pre-configured ways to slice & filter data. You can return multiple metrics at once or just a single metric. You can filter & group data in a variety of ways. All data is returned in JSON. Note that this API is concerned with filtering events.

Analytics API: docs.datasnapioentityapi.apiary.io

Use the Entity API to pull back the entity ID’s to use in the Analytics API call. For finding and retrieving Entities such as place/Beacons please use our Entity API located at: docs.datasnapioentityapi.apiary.io


Troubleshooting

I’m seeing data in my dashboard KPI’s, but I’m not seeing beacons or places

  • Are you using the entity API to send in data? If you are you must send entities in separately

Paths and visits are not showing up in my dashboard

  • Check that you are sending global_distinct_id with each event. You must include this or data will not show in reporting.

I’m not seeing any geofence visits

  • A geofence visit is denoted by a geofence_depart event. Make sure you are sending geofence_depart events

I’m not seeing any data in my GPS User Heatmap

  • This widget shows the GPS position of the user and should not to be confused with the Lat / Long of the beacon. Make sure you are sending user lat / long along with the event.

"global_position": { "location": { "coordinates": [ 37.7833, -122.4167 ] }, "altitude": 13, "accuracy": 10, "course": 77.34375, "speed": 4 }



Latest Release (08/10/2016)

This is the Datasnap SDK Release 2.0.0 for Android and iOS.

Datasnap iOS SDK
Datasnap Android SDK

Development Improvements


  • Apps integrating with Gimbal can leverage a pre-defined method that sends default events to the Datasnap SDK with a quick turnkey solution.

  • Pre-defined list of event-type methods lets developers quickly send an event with default configurations.

  • Improved logging and troubleshooting.

  • Configurable batch sizing and timeout threshold.

  • Event compression for better app performance.


Data Collection Improvements


  • Offline caching collects data when the user goes offline and sends it after they come back online.

  • Data collection for OS, Version and manufacturer information improves reporting.

  • Data collection is defaulted to on for Google Ad ID/Hashed email/Device info for Android and Hashed email/Device info for iOS, so long as a user has allowed their device to collect that information. If your privacy consent with end users does not allow sharing this data with Neustar, you MUST override the default setting.


Prior Releases

Release 2016-06-17

Release 2016-04-12

Release 2016-01-14

Release 2016-01-04

Release 2015-12-07

Release 2015-08-26

Release 2015-07-14