Friday, December 19, 2014

Parsing BLE advertisement packets

Ever since I created the Gas Sensor demo (post here, video here, presentation here), I had the feeling of an unfinished business. That demo sent the sensor data in BLE advertisement packets so the client never connected to the sensor but received data from the sensor in a broadcast-like fashion. The implementation looked like this:

        public void onLeScan(final BluetoothDevice device, int rssi, byte[] scanRecord) {
            String deviceName = device.getName();
...
                int addDataOffs = deviceName.length() + 16;
                int siteid = ((int)scanRecord[addDataOffs]) & 0xFF;
                int ad1 = ((int)scanRecord[addDataOffs+1]) & 0xFF;

This was a quick & dirty solution that remained there from my earliest prototypes. It sort of assumes that the structure of the BLE advertisement packet is fixed so the sensor data can always be found at fixed locations of the advertisement packet. This does not have to be the case, Bluetooth 4.0 Core Specification, Part C, Appendix C (or Core Specification Supplement in case of 4.2 version) describes, how the fields of the advertisement packets look like. It just so happens that with the given version of the RFDuino BLE module, the Manufacturer Specific Data field where RFDuino puts the user data for the advertisement packet can always be found at a specific location.

The proper way is of course to parse this data format according to the referred appendix of the specification and in this post I will show you how I implemented it.

Here are the three example programs mentioned in this post: blescan.zip, gassensordemo.zip, gas_adv.ino

Let's see first the BLEScan project.

Update: the project has been updated to support more of the 4.2 elements. It has also been converted into an Android Studio project but the download material contains only the app/src part of the tree.

Update: I was asked by e-mail, how to import the project (in blescan.zip) into Android Studio. Here is a simple process.
  • Create a new project in Android Studio under any name. Make sure that your project supports at least API level 18. Choose the "create no Activity" option.
  • Once your project is created, go and find it on the disk. On my Ubuntu system, the project files go under ~/StudioProjects/<ProjectName> where <ProjectName> is the name you gave to your project. We will call this directory <ProjectDir>.
  • Go into <ProjectDir>/app/src and delete everything there. Copy blescan.zip into <ProjectDir>/app/src and unzip it. It will create a single directory called "main" and the sources below.
  • In Android Studio, do File/Synchronize. After that is completed, you can open your project files, build APK, etc.

The parser code is under the hu.uw.pallergabor.ble.adparser package. Then you just give the scanRecord array to AdParser's parseAdData method like this:

            ArrayList<AdElement> ads = AdParser.parseAdData(scanRecord);

and then you get an array of objects, each describing an element in the scan record. These objects can also produce printable representation like this:



Now let's see the revised GasSensorDemo project, how the gas sensor measurement is properly parsed out of the scan record. First we parse the scan packet fields:

                ArrayList<AdElement> ads = AdParser.parseAdData(scanRecord);

Then we look for a TypeManufacturerData element which corresponds to a Manufacturer Specific Data field in BLE. We make an extra check to make sure that the manufacturer field in the Manufacturer Specific Data is 0x0000 because RFDuino always creates a Manufacturer Specific Data field like that if the application programmer specifies additional advertisement data.

                    AdElement e = ads.get(i);
                    if( e instanceof TypeManufacturerData ) {
                        TypeManufacturerData em = (TypeManufacturerData)e;
                        if( em.getManufacturer() == 0x0000) {


It would be tempting to use a custom manufacturer field or better, a Service Data field. But then we run into another limitation of RFDuino because RFDuino with its default firmware is only able to create advertisement packets like in the previous example. This is not bad because it allows the programmer to achieve quick success but later on, we will need more flexibility and that will need another BLE module.

4 comments:

Anonymous said...

I've build a different demo with a temperature sensor that also sends it over advertisements. But I ran into the same issues as you did when i tried to add more data.
Did you try and look at this ble guide? it explained to me exactly what I needed to know.

The page isn't enough. You still need to look a t the Bluetooth spec.

Gabor Paller said...

Thanks, Damon. You may want to check out the updated code, it supports more BLE advertisement elements (although still not all the 4.2 elements).

Unknown said...

Gabor -
I'm trying to capture BLE broadcast (ADV_NONCONN_IND) packets inside of my android app. However I'm not able to receive any packets.

I know my peripheral device is sending packets because I have another BLE capture device that sees all the packets. However my app inside the phone doesn't see any BLE packets.

My app can see other packets such as ADV_SCAN_ID but not ADV_NONCONN_IND (broadcast) packets.

public void uploadScanBytes(SensorDataUploader sensorDataUploader, int count) {
BluetoothAdapter btAdapter = getBluetoothAdapter();
if (btAdapter == null) return;

BluetoothLeScanner scanner = btAdapter.getBluetoothLeScanner();
ScanSettings settings = new ScanSettings.Builder()
.setScanMode(ScanSettings.SCAN_MODE_LOW_LATENCY)
.setCallbackType(ScanSettings.CALLBACK_TYPE_ALL_MATCHES)
.build();
scanner.startScan(Collections.emptyList(), settings, new LimitedScanRecordReader(sensorDataUploader, count, scanner));
}


public void onScanResult(int callbackType, ScanResult result) {

if(result.getDevice().getAddress().equals("00:AB:2c:A1:E2:F1")) {
long timestamp = System.currentTimeMillis() - SystemClock.elapsedRealtime() +
result.getTimestampNanos() / 1000000;
byte[] rawBytes = result.getScanRecord().getBytes();
Log.i(DataTransferService.class.getName(), "Raw bytes: " + byteArrayToHex(rawBytes));
sensorDataUploader.upload(timestamp, rawBytes);
}
}

Please let me know what I'm doing wrong.
Thanks.

Gabor Paller said...

Tony, I wrote an e-mail to you about the issue. Could you try the Android application linked to this post? This application is very similar to yours (uses BluetoothLeScanner) but is reliably able to capture unidirectional advertisement packet indications that my sensors generate. Look for the onScanResult method in BLESensorGWService.java, there's a log message there.