mædoc's notes

elf loading on esp32

the idea

ESP provides an elf_loader, which allows dynamically loading a distinct binary from some data source and running it. There's a quick README and a two part example

and the implementation here.

"app" in scarequotes because this isn't like dlsym on linux, it seems you can only really call a main.

where does the elf come from?

in the xample, the elf is embedded in the binary by the idf_component_register CMake command,

set(TEST_ELF "test_riscv.elf")

idf_component_register(SRCS "elf_loader_example_main.c"
                       INCLUDE_DIRS ""
                       EMBED_TXTFILES ${TEST_ELF})

which gives main.c access to the array of bytes,

extern const uint8_t test_elf_start[] asm("_binary_test_riscv_elf_start");

so I guess any filename like foo_bar.baz becomes a variable named _binary_foo_bar_baz_start.

the api

from the header

/* @brief Initialize ELF object. */
int esp_elf_init(esp_elf_t *elf);

/* @brief Decode and relocate ELF data. */
int esp_elf_relocate(esp_elf_t *elf, const uint8_t *pbuf);

/* @brief Request running relocated ELF function. */
int esp_elf_request(esp_elf_t *elf, int opt, int argc, char *argv[]);

so while the example embeds the elf binary, we could just as well load it from an NVS storage blob, a partition, from wifi, from a scanned qr code... the main thing to keep in mind is that (a) we can't access arbitrary symbols, just a main(argc, argv) and (b) the build system requires the elf to be a separate project.

an example elf project

following the instructions in the docs about setting it up, I have two elf projects suitably named foo & bar, with a layout like

├── bar
│   ├── CMakeLists.txt
│   ├── dependencies.lock
│   ├── main
│   │   ├── bar.c
│   │   ├── CMakeLists.txt
│   │   └── idf_component.yml
│   └── sdkconfig

with CMakeLists.txt

cmake_minimum_required(VERSION 3.16)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(bar)
include(elf_loader)
project_elf(bar)

and bar/main/bar.c which will start a task (since, ideally, if we can't call an elf's symbols we can still work with it via freertos/esp apis), printing stuff,

#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <esp_log.h>

static const char *TAG = "bar";

static void f(void *arg) {
  int count = 0;
  while (1) {
    ESP_LOGW(TAG, "count = %d", count++);
    vTaskDelay(pdMS_TO_TICKS(512));
  }
  vTaskDelete(NULL);
}

int main(int arg, char *argv[])
{
  xTaskCreate(f, "barf", 2048, NULL, 5, NULL);
  return 0;
}

which builds with idf.py elf generating quite a small thing,

$ ls -lh bar/build/*.app.elf
-rwxr-xr-x 1 duke duke 1.4K Apr 14 18:23 bar/build/bar.app.elf

that the elfs are potentially small means we can content hash them in NVS blob storage for a flexible way to update modules, etc.

loading dem elves

so while it'll be cool to load elf binaries from outer space or wherever, let's take the demo approach of embedding them,

idf_component_register(SRCS "main.c" EMBED_TXTFILES "foo.app.elf" "bar.app.elf" )

then in main.c, something like this,

extern const uint8_t bar_elf_start[] asm("_binary_bar_app_elf_start");

static void run_elf(const uint8_t *elf_bytes) {
  esp_elf_t elf;
  ESP_ERROR_CHECK(esp_elf_init(&elf));
  ESP_ERROR_CHECK(esp_elf_relocate(&elf, elf_bytes));
  ESP_ERROR_CHECK(esp_elf_request(&elf, 0, 0, NULL));
}

static void run_bar(void *arg) {
  ESP_LOGW(TAG, "trying to start bar");
  run_elf(bar_elf_start);
  vTaskDelete(NULL);
}

but when running it, the elf loader can't resolve some symbols like,

W (313) main: trying to start bar
I (313) ELF: ELF loader version: 1.0.0
I (323) ELF: Too much padding before segment[2], padding: 4243
I (323) ELF: elf->entry=0x408128d8

E (333) ELF: Can't find common esp_log_write
ESP_ERROR_CHECK failed: esp_err_t 0xffffffa8 (ERROR) at 0x4200c176

resolving symbols?

hm, interesting comment here, int he cmake for the elf project (not the loader)

# Set other components to link to the ELF file
# e.g: set(ELF_COMPONENTS "log" "esp_wifi")

so if we do that (add just log for now, since we're not using anything else) and rebuild, we find there's other stuff it can't find like vprintf. What's that cmake variable ELF_COMPONENTS up to anyway? well, let's see,

    list(PREPEND ELF_COMPONENTS "main")
    if(ELF_COMPONENTS)
        foreach(c ${ELF_COMPONENTS})
            list(APPEND elf_libs "esp-idf/${c}/lib${c}.a")
            list(APPEND elf_dependeces "idf::${c}")
        endforeach()

let's go fishing..

so we need to maybe get a few more things in there..

set(ELF_COMPONENTS "log" "c")

results in

ninja: error: 'idf::c', needed by 'elf_app', missing and no known rule to make it

hm, the libc.a is somewhere (since idf.py size-components will list it) but not clear how to get tell the toolchain to add it. Maybe I could dig into the cmake a bit, but I think it's worth filing a bug

https://github.com/espressif/esp-iot-solution/issues/497

let's see what happens

this is a work in progress with code here.