WiFiBoy32 + WB32-SFX BLE-MIDI 數位音源實驗成功!
-
2017-10-28 by Derek w/ESP32徹底研究小組
最近正在開發中的 WB32-SFX 真的是一個很有趣的 ESP32 應用開發板,除了可以做出 WiFi Web Radio 接收網路廣播,解 Live Streaming,也能變成藍牙喇叭接收器,或是透過 microSD 卡,播放 mp3/aac/ogg 或 midi 的音樂,讓 Ricky 的 HiFiBoy 計畫變得更加有趣。
WB32-SFX 的 VS-1053B 晶片內建有 General MIDI 音源,可以發出一百多種樂器的聲音,只要能送進 MIDI Type0 的檔案或即時的 MIDI 訊號,就能演奏出美妙的樂音。
我玩過好一陣子的數位音樂研究,其實非常熟悉 MIDI 的規格,二十年前曾經參與過 MIDI Wavetable Synthesizer 晶片開發,做過 MIDI Sequencer 播放器,也曾經研究 MIDI 音色製作,還參與編製過數千首 MIDI 卡拉OK的音樂。不過,這些只是好漢愛提的當年勇,我對這兩年剛出爐的藍牙無線版 BLE-MIDI 規格卻是一無所知,從未有過任何使用經驗。
下載規格書:BLE-MIDI Specification @ Midi.Org
為了 WB32-SFX 的 Realtime MIDI 音源應用範例,我們「ESP32 徹底研究小組」決定進行一個「無線藍牙 BLE-MIDI 收發器」的實作計畫,讓內建 WB32-SFX 模組的「HiFiBoy 可程式化喇叭」也可以接收無線 BLE-MIDI 控制器的指令,搖身一變成為一個厲害的數位樂器音源主機,可以用來即時演奏音樂。
這確實是一個超級有趣的實作計畫,但似乎也是個難度相當高的實驗,畢竟 ESP32 的 BLE 範例都只是些小測試程式,似乎還沒有什麼應用範例可以參考。
不管如何,再多難關總是要一關一關過的。
我們先在 Amazon.jp 訂購了兩個 Korg 的 microKEY Air 藍牙 MIDI 鍵盤來做實驗。
配對接上 iPad 的 GarageBand 或 Korg Gadget/Module App,就可以變成好玩的數位音樂錄音室。
App 雖然好好玩,但我們的挑戰還沒開始呢!這個週末的實作計畫有三個:
1. 研究 MIDI BLE 規格,弄清楚 microKEY Air 如何連上 iPad 的 GarageBand 進行彈奏。
2. 寫一個模擬 MIDI BLE 控制器的程式,讓 WiFiBoy32 變成 microKEY Air,可以無線連接 iPad 上的 MIDI App 彈奏發出聲音。
3. 寫一個能接收 MIDI BLE 控制器訊號的程式,讓 WB32-SFX 能接收 microKEY Air 的琴鍵 MIDI 訊號,彈奏出鋼琴樂音。
第一個計畫很快有了初步的理解,我們下載了 MIDI.ORG 的 BLE-MIDI 規格書,只有區區十頁資料。仔細研究了幾個小時,發現重點就是兩個:
1. BLE-MIDI 的設備,一定會有一個 MIDI Service, UUID: 03B80E5A-EDE8-4B33-A751-6CE34EC4C700
2. 要獲取 MIDI Service 的訊號,需要透過 MIDI Data I/O Characteristic UUID: 7772E5DB-3868-4112-A1A9-F2669D106BF3
3. BLE MIDI 資料格式:前面兩個 Timestamp bytes,後面接著 3 個 bytes 或更多的 MIDI Codes。
這規格其實沒有太大的問題,比較麻煩的就是要先弄懂 BLE 的幾個規矩:
1. BLE 透過 GATT 規範,由 Client 主機端向設備端的 Server 接收資料。沒看錯,是 Client 主機端,Server 設備端。也就是說 iPad 是 GATT Client,microKEY Air 是 GATT Server。
2. GATT Server 要 Advertise 廣播適當的資料給大家看,讓 GATT Client 能找到 MIDI Service 與 Data I/O Characteristic。MIDI Service 需要提供 Write, Read, Notify 等三種服務。
3. GATT Client 要 Scan 所有的 BLE 設備,選一個有 MIDI Service UUID 的來連接。再打開其 Data I/O Characteristic,建立 Notify 訊息 Callback。就可以從收到的 Notification 來解出 MIDI Codes。
花了一些時間弄清楚了 BLE 與 MIDI Service 的協定流程,就可以開始寫程式了。
我們以 ESP32 Arduino 寫了第一個實驗程式,WB32-GATT-Server,先裝在 WiFiBoy32 開發板上,透過 WiFiBoy32 上的六個按鍵,可以接上 iPad 的 GarageBand 演奏鋼琴。
這個程式寫得非常順利,成果請看這個影片:
這個程式只有三百多行,原始碼在此: wb32_gatt_server_ble_midi.ino
<p>/* * GATT-Server for BLE-MIDI -- WiFiBoy32 Demo Code for Arduino/ESP32 * * Oct 28, 2017 (derek@wifiboy.org) * Modified from http://www.iotsharing.com/2017... */ #pragma GCC diagnostic push #pragma GCC diagnostic warning "-fpermissive" #include <stdio.h> #include <stdlib.h> #include <string.h> #include "freertos/FreeRTOS.h" #include "freertos/task.h" #include "freertos/event_groups.h" #include "esp_system.h" #include "esp_log.h" #include "nvs_flash.h" #include "bt.h" #include "bta_api.h" #include "esp_gap_ble_api.h" #include "esp_gatts_api.h" #include "esp_bt_defs.h" #include "esp_bt_main.h" #include "esp_bt_main.h" #include "sdkconfig.h" #include "wifiboy32.h" #pragma GCC diagnostic pop</p> <p>/* this function will be invoked to handle incomming events */ static void gatts_profile_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, esp_ble_gatts_cb_param_t *param);</p> <p>#define TEST_DEVICE_NAME "MiDiBoyBLE!"</p> <p>/* maximum value of a characteristic */ #define GATTS_DEMO_CHAR_VAL_LEN_MAX 0xFF</p> <p>/* value range of a attribute (characteristic) */ uint8_t attr_str[] = {0x00}; esp_attr_value_t gatts_attr_val = { .attr_max_len = GATTS_DEMO_CHAR_VAL_LEN_MAX, .attr_len = sizeof(attr_str), .attr_value = attr_str, };</p> <p>/* service uuid */ static uint8_t service_uuid128[32] = { 0x00, 0xC7, 0xC4, 0x4E, 0xE3, 0x6C, 0x51, 0xA7, 0x33, 0x4B, 0xE8, 0xED, 0x5A, 0x0E, 0xB8, 0x03, 0xF3, 0x6B, 0x10, 0x9D, 0x66, 0xF2, 0xA9, 0xA1, 0x12, 0x41, 0x68, 0x38, 0xDB, 0xE5, 0x72, 0x77 };</p> <p>static uint8_t raw_adv_data[] = { 0x02, 0x01, 0x06, 0x11, 0x07, 0x00, 0xC7, 0xC4, 0x4E, 0xE3, 0x6C, 0x51, 0xA7, 0x33, 0x4B, 0xE8, 0xED, 0x5A, 0x0E, 0xB8, 0x03, }; static uint8_t raw_scan_rsp_data[] = { 0x0c, 0x09, 'M','i','D','i','B','o','y','B','L','E','!' };</p> <p>esp_ble_adv_params_t test_adv_params;</p> <p>#define PROFILE_ON_APP_ID 0 #define CHAR_NUM 1</p> <p>struct gatts_characteristic_inst{ esp_bt_uuid_t char_uuid; esp_bt_uuid_t descr_uuid; uint16_t char_handle; esp_gatt_perm_t perm; esp_gatt_char_prop_t property; uint16_t descr_handle; };</p> <p>struct gatts_profile_inst { esp_gatts_cb_t gatts_cb; uint16_t gatts_if; uint16_t app_id; uint16_t conn_id; uint16_t service_handle; esp_gatt_srvc_id_t service_id; struct gatts_characteristic_inst chars[CHAR_NUM]; };</p> <p>/* One gatt-based profile one app_id and one gatts_if, this array will store the gatts_if returned by ESP_GATTS_REG_EVT */ static struct gatts_profile_inst test_profile;</p> <p>/* this callback will handle process of advertising BLE info */ static void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) { switch (event) { case ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT: esp_ble_gap_start_advertising(&test_adv_params); break; case ESP_GAP_BLE_ADV_DATA_RAW_SET_COMPLETE_EVT: esp_ble_gap_start_advertising(&test_adv_params); break; case ESP_GAP_BLE_SCAN_RSP_DATA_RAW_SET_COMPLETE_EVT: esp_ble_gap_start_advertising(&test_adv_params); break; case ESP_GAP_BLE_ADV_START_COMPLETE_EVT: //advertising start complete event to indicate advertising start successfully or failed if (param->adv_start_cmpl.status != ESP_BT_STATUS_SUCCESS) Serial.println("Advertising start failed\n"); break; case ESP_GAP_BLE_ADV_STOP_COMPLETE_EVT: if (param->adv_stop_cmpl.status != ESP_BT_STATUS_SUCCESS) Serial.println("Advertising stop failed\n"); else Serial.println("Stop adv successfully\n"); break; default: break; } }</p> <p>/* this callback handle BLE profile such as registering services and characteristics, send response to central device */ static void gatts_profile_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, esp_ble_gatts_cb_param_t *param) { switch (event) { /* create service event */ case ESP_GATTS_REG_EVT: printf("REGISTER_APP_EVT, status %d, app_id %d\n", param->reg.status, param->reg.app_id); test_profile.service_id.is_primary = true; test_profile.service_id.id.inst_id = 0x00; test_profile.service_id.id.uuid.len = ESP_UUID_LEN_128; for(int i=0; i<16; i++) test_profile.service_id.id.uuid.uuid.uuid128[i] = service_uuid128[i]; esp_ble_gap_set_device_name(TEST_DEVICE_NAME); esp_ble_gap_config_adv_data_raw(raw_adv_data, sizeof(raw_adv_data)); esp_ble_gap_config_scan_rsp_data_raw(raw_scan_rsp_data, sizeof(raw_scan_rsp_data)); esp_ble_gatts_create_service(gatts_if, &test_profile.service_id, 4); break; /* when central device request info from this device, this event will be invoked and respond */ case ESP_GATTS_READ_EVT: { printf("ESP_GATTS_READ_EVT, conn_id %d, trans_id %d, handle %d\n", param->read.conn_id, param->read.trans_id, param->read.handle); esp_gatt_rsp_t rsp; memset(&rsp, 0, sizeof(esp_gatt_rsp_t)); rsp.attr_value.handle = param->read.handle; rsp.attr_value.len = 1; rsp.attr_value.value[0] = 0; esp_ble_gatts_send_response(gatts_if, param->read.conn_id, param->read.trans_id, ESP_GATT_OK, &rsp); break; } /* when central device send data to this device, this event will be invoked */ case ESP_GATTS_WRITE_EVT: { printf("ESP_GATTS_WRITE_EVT, conn_id %d, trans_id %d, handle %d\n", param->write.conn_id, param->write.trans_id, param->write.handle); printf("value len %d, value %08x\n", param->write.len, *(uint8_t param->write.value); break; } /* start service and add characterstic event */ case ESP_GATTS_CREATE_EVT: printf("status %d, service_handle %d\n", param->create.status, param->create.service_handle); test_profile.service_handle = param->create.service_handle; esp_ble_gatts_add_char(test_profile.service_handle, &test_profile.chars[0].char_uuid, ESP_GATT_PERM_READ | ESP_GATT_PERM_WRITE, ESP_GATT_CHAR_PROP_BIT_READ | ESP_GATT_CHAR_PROP_BIT_WRITE_NR | ESP_GATT_CHAR_PROP_BIT_NOTIFY, &gatts_attr_val, NULL); esp_ble_gatts_start_service(test_profile.service_handle); break; case ESP_GATTS_ADD_CHAR_EVT: { printf("ADD_CHAR_EVT, status %d, attr_handle %d, service_handle %d (%x)\n", param->add_char.status, param->add_char.attr_handle, param->add_char.service_handle, param->add_char.char_uuid.uuid.uuid128[0]); if(param->add_char.char_uuid.uuid.uuid128[0] == 0xf3){ test_profile.chars[0].char_handle = param->add_char.attr_handle; } break; } case ESP_GATTS_ADD_CHAR_DESCR_EVT: printf("ESP_GATTS_ADD_CHAR_DESCR_EVT, status %d, attr_handle %d, service_handle %d\n", param->add_char.status, param->add_char.attr_handle, param->add_char.service_handle); break; case ESP_GATTS_DISCONNECT_EVT: esp_ble_gap_start_advertising(&test_adv_params); wb32_fillRect(20,120,120,15,0); break; case ESP_GATTS_CONNECT_EVT: { printf("ESP_GATTS_CONNECT_EVT\n"); esp_ble_conn_update_params_t conn_params = {0}; memcpy(conn_params.bda, param->connect.remote_bda, sizeof(esp_bd_addr_t)); /* For the IOS system, please reference the apple official documents about the ble connection parameters restrictions. */ conn_params.latency = 0; conn_params.max_int = 0x30; conn_params.min_int = 0x20; conn_params.timeout = 500; printf("ESP_GATTS_CONNECT_EVT, conn_id %d, remote %02x:%02x:%02x:%02x:%02x:%02x:, is_conn %d\n", param->connect.conn_id, param->connect.remote_bda[0], param->connect.remote_bda[1], param->connect.remote_bda[2], param->connect.remote_bda[3], param->connect.remote_bda[4], param->connect.remote_bda[5], param->connect.is_connected); test_profile.conn_id = param->connect.conn_id; //start sent the update connection parameters to the peer device. esp_ble_gap_update_conn_params(&conn_params); wb32_drawString("Connected!", 20, 120, 1, 2); break; } default: break; } }</p> <p>static void gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, esp_ble_gatts_cb_param_t *param) { /* If event is register event, store the gatts_if for each profile */ if (event == ESP_GATTS_REG_EVT) { if (param->reg.status == ESP_GATT_OK) test_profile.gatts_if = gatts_if; else { printf("Reg app failed, app_id %04x, status %d\n", param->reg.app_id, param->reg.status); return; } } /* here call each profile's callback */ if (gatts_if == ESP_GATT_IF_NONE || gatts_if == test_profile.gatts_if) if (test_profile.gatts_cb) test_profile.gatts_cb(event, gatts_if, param); }</p> <p>void setup() { wb32_init(); wb32_setTextColor(wbYELLOW, wbYELLOW); wb32_drawString("MiDiBoy on WiFiBoy", 20, 20, 1, 3); wb32_setTextColor(wbWHITE, wbWHITE); wb32_drawString("BLE-MIDI Test", 20, 70, 1, 2); wb32_setTextColor(wbGREEN, wbGREEN); wb32_drawString("(C)2017 WiFiBoy.Org & WiFiBoy.Club", 20, 300, 2, 1); for(int i=0; i<7; i++) wb32_fillRect(i*31+13, 160, 28, 120, 0xffff); for(int i=0; i<2; i++) wb32_fillRect(i*31+31, 160, 23, 75, 0); for(int i=3; i<6; i++) wb32_fillRect(i*31+31, 160, 23, 75, 0); Serial.begin(115200); test_adv_params.adv_int_min = 0x20; test_adv_params.adv_int_max = 0x30; test_adv_params.adv_type = ADV_TYPE_IND; test_adv_params.own_addr_type = BLE_ADDR_TYPE_PUBLIC; test_adv_params.channel_map = ADV_CHNL_ALL; test_adv_params.adv_filter_policy = ADV_FILTER_ALLOW_SCAN_ANY_CON_ANY; test_profile.gatts_cb = gatts_profile_event_handler; test_profile.gatts_if = ESP_GATT_IF_NONE; /* Not get the gatt_if, so initial is ESP_GATT_IF_NONE */ test_profile.chars[0].char_uuid.len = ESP_UUID_LEN_128; for(int i=0; i<16; i++) test_profile.chars[0].char_uuid.uuid.uuid128[i] = service_uuid128[i+16]; test_profile.chars[0].perm = ESP_GATT_PERM_READ | ESP_GATT_PERM_WRITE_ENCRYPTED; test_profile.chars[0].property = ESP_GATT_CHAR_PROP_BIT_READ | ESP_GATT_CHAR_PROP_BIT_WRITE_NR | ESP_GATT_CHAR_PROP_BIT_NOTIFY; btStart(); esp_bluedroid_init(); esp_bluedroid_enable(); esp_ble_gatts_register_callback(gatts_event_handler); esp_ble_gap_register_callback(gap_event_handler); esp_ble_gatts_app_register(0); pinMode(33,INPUT); pinMode(17,INPUT); pinMode(27,INPUT); pinMode(32,INPUT); pinMode(34,INPUT); pinMode(35,INPUT); }</p> <p>int lastkey1=0, lastkey2=0, lastkey3=0, lastkey4=0, lastkey5=0, lastkey6=0;</p> <p>void loop() { char mididata1[]={0x82, 0x84, 0x90, 0x30, 0x50}; char mididata2[]={0x82, 0x84, 0x80, 0x30, 0x50}; if ((digitalRead(33)==0)&&(lastkey1!=1)) { mididata1[3]=0x30; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata1, false); lastkey1=1; wb32_fillRect(0*31+20, 250, 14, 15, wbYELLOW); } if ((digitalRead(33)==1)&&(lastkey1==1)) { mididata2[3]=0x30; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata2, false); lastkey1=0; wb32_fillRect(0*31+20, 250, 14, 15, wbWHITE); } if ((digitalRead(17)==0)&&(lastkey2!=2)) { mididata1[3]=0x32; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata1, false); lastkey2=2; wb32_fillRect(1*31+20, 250, 14, 15, wbGREEN); } if ((digitalRead(17)==1)&&(lastkey2==2)) { mididata2[3]=0x32; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata2, false); lastkey2=0; wb32_fillRect(1*31+20, 250, 14, 15, wbWHITE); } if ((digitalRead(27)==0)&&(lastkey3!=3)) { mididata1[3]=0x34; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata1, false); lastkey3=3; wb32_fillRect(2*31+20, 250, 14, 15, wbBLUE); } if ((digitalRead(27)==1)&&(lastkey3==3)) { mididata2[3]=0x34; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata2, false); lastkey3=0; wb32_fillRect(2*31+20, 250, 14, 15, wbWHITE); } if ((digitalRead(32)==0)&&(lastkey4!=4)) { mididata1[3]=0x35; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata1, false); lastkey4=4; wb32_fillRect(3*31+20, 250, 14, 15, wbRED); } if ((digitalRead(32)==1)&&(lastkey4==4)) { mididata2[3]=0x35; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata2, false); lastkey4=0; wb32_fillRect(3*31+20, 250, 14, 15, wbWHITE); } if ((digitalRead(34)==0)&&(lastkey5!=5)) { mididata1[3]=0x37; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata1, false); lastkey5=5; wb32_fillRect(4*31+20, 250, 14, 15, wbBLUE); } if ((digitalRead(34)==1)&&(lastkey5==5)) { mididata2[3]=0x37; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata2, false); lastkey5=0; wb32_fillRect(4*31+20, 250, 14, 15, wbWHITE); } if ((digitalRead(35)==0)&&(lastkey6!=6)) { mididata1[3]=0x39; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata1, false); lastkey6=6; wb32_fillRect(5*31+20, 250, 14, 15, wbYELLOW); } if ((digitalRead(35)==1)&&(lastkey6==6)) { mididata2[3]=0x39; esp_ble_gatts_send_indicate(test_profile.gatts_if, test_profile.conn_id, test_profile.chars[0].char_handle, 5, mididata2, false); lastkey6=0; wb32_fillRect(5*31+20, 250, 14, 15, wbWHITE); } delay(10); }</p>
第二個實驗程式,是要讓 WB32-SFX 能接收 microKEY Air 的琴鍵 MIDI 訊號。
這個程式遇到比較多的麻煩,沒有興趣深究的不需要仔細看,摘要如下:
1. 參考 ESP-IDF 範例寫了基本的 BLE GATT-Client 程式,可以接收先前 WiFiBoy32 的 GATT-Server 按鍵 MIDI Codes,但收不到 microKEY Air 的按鍵 MIDI Codes。
2. 花了些時間才發現是 Notify 的設定,需要對 Descriptor 寫入一個 0x01 的數值,必須正確設定 GATT Authentication Type,要選用 NO_MITM 的認證設定。
3. ESP32 的 BLE 功能是要開 Bluetooth Dual Mode 的,整個模組耗電流實測高達 160mA(USB 5V端),根本就不太 Low Energy 啊!ESP32 模組甚至還會發出微微溫度,顯然還需要繼續努力,Espressif 團隊說省電模式正在開發中,現在的 3.0 版 ESP-IDF 似乎還沒有太大改進。我試著將 ESP32 主 CPU 速度從 240Mhz 降為 80Mhz,整體耗電流可以降至 130mA 左右,這才使得 WB32-SFX 電力輸出有點虛弱的工程樣板,可以正常運作起來。ESP32 的 WiFi+BLE 無線通訊模組的瞬間電流需求相當高,據說會高達 300~400mA,WB32 開發板需要特別加強瞬間供電的能力。
總之,花了幾個小時,第二個程式終於可以接收 microKEY Air 的 MIDI 訊號,讓 WB32-SFX 發出令人感動的鋼琴聲了!
請看影片:(稍後上傳)
原始程式在這裡(稍後上傳),若有興趣研究細節,請看程式裡的註解,或是在本文後面留言討論。
做這個範例的過程雖然辛苦,但實驗的結果一路都非常好玩。
這次的實作成果會在下週末的 Maker Faire Taipei 2017,11/3~11/5 在光華商場旁的華山園區展出。我們 WiFiBoy.Org 有兩個攤位,攤位名稱「HiFiBoy DIy Speaker」,戶外是在: 室內則是:。歡迎蒞臨指導!
(Maker Faire Taipei 展覽期間有這次實驗用的 WiFiBoy32 的上市特惠方案,現場數量有限,只有 100 套,敬請支持!)