I’ve been using the Rust Safe APIs for the past few weeks and steadily move towards an idea of how all moving parts come together. As a kind of exercise I’ve worked on improving and updating the bindings to Node.js. This post serves as memory and reference to myself and anyone interested. (I welcome any feedback from someone proficient with Rust!)
A major challenge I’m trying to wrap my head around, is what I assume is due to the Rust ownership model and its opposite: the garbage collector of Node.js.
Node.js has an API (N-API) that allows us to translate Rust into JavaScript. For example, one can turn an i32 into a Number. Say we have the following Safe idiom in Rust:
#[tokio::main]
async fn main() {
let mut safe = sn_api::Safe::new(None, std::time::Duration::from_secs(120));
safe.connect(None, None, None).await.unwrap();
let (url, kp) = safe.keys_create_preload_test_coins("9").await.unwrap();
}
Then the equivalent in idiomatic Node.js could be this:
const { Safe } = require('sn_api');
(async () => {
const safe = new Safe(null, 120);
await safe.connect();
const [url, kp] = await safe.keys_create_preload_test_coins('9');
})().catch(r => console.dir(r));
With N-API, these kinds of translations can be implemented.
napi-rs — N-API in Rust
napi-rs is a Rust library offering an ergonomic way to use N-API. I’ll use this to translate the Rust API into JS constructs.
N-API has a way to wrap a native (Rust) object (e.g. sn_api::Safe) into a JS object. When a method is called, the native instance can then be borrowed (unwrapped) and used.
Now let me try to explain my problem. If we want to do something asynchronously (e.g. fetch a file from the network), then we need a runtime for this. (The Safe APIs mostly use async functions that have to be ‘polled to completion’ by a runtime.) Node.js only has a single main (event loop) thread, thus any function we create should not block. If it would block, then we might be blocking the whole application from executing anything else.
Imagine we create a simple JavaScript class that wraps around an integer (e.g. i32). napi-rs has a convenience function that allows us to execute a future in a Tokio runtime. This function makes sure there is only one runtime instantiated for our addon, run that runtime on a separate thread (independent of Node.js main thread), and send our futures to that runtime, handling it and turning it into a JavaScript Promise. A very convenient function indeed.
Let me illustrate a problem I could bump into:
#[js_function(0)]
fn constructor(ctx: CallContext) -> Result<JsUndefined> {
let mut this: JsObject = ctx.this_unchecked();
ctx.env.wrap(&mut this, 572)?; // Wrapping the native Rust value.
ctx.env.get_undefined()
}
#[js_function(1)]
fn my_method(ctx: CallContext) -> Result<JsUndefined> {
let this: JsObject = ctx.this_unchecked();
let num: &mut i32 = ctx.env.unwrap(&this)?; // Borrowing the value.
// Convenience method to execute futures, returning a Promise!
ctx.env.execute_tokio_future(
async {
println!("{}", num); // Compile error!!!
// (because reference might be invalid)
},
|&mut env, val| {
ctx.env.get_undefined()
},
)
}
The native Rust value can be unwrapped/borrowed, but I can’t pass it to the Tokio runtime on the other thread, because the data it references might have been garbage collected (I assume) in Node.js. The actual error I am getting is the following:
explicit lifetime required in the type of `ctx`
lifetime `'static` required
Again assuming: the context that provides the borrow might be dropped entirely, thus dropping the native Rust value.
One solution is to clone the value and move it into the async function. With an integer, that seems like a good solution. But, with sn_api::Safe I am not so sure about it. If the object is cloned and we call, for example, connect(), then only the cloned Safe instance will be connected, no the one still wrapped in the JS object.
(I’ve used this solution for now in a draft I’m making here.)
To be continued.
PS: I updated the existing bindings (they use `neon`) for a bit and described some similar problems [here](https://github.com/maidsafe/sn_nodejs/pull/85#issuecomment-779909946). I'm starting to think cloning is a sensible solution to most problems in Node.js. But that also means that the Safe Rust APIs should be up to that task and allow it to work that way.
. I really don’t know what this makes possible, but it seems important.
. These are steps towards a WASI 1.0 W3C Standard.