erl_nif

API functions for an Erlang NIF library

Warning!

The NIF concept was introduced in R13B03 as an EXPERIMENTAL feature. The interfaces may be changed in any way in coming releases. The plan is however to lift the experimental label and maintain interface backward compatibility from R14B.

Incompatible changes in R13B04:

  • The function prototypes of the NIFs have changed to expect argc and argv arguments. The arity of a NIF is by that no longer limited to 3.
  • enif_get_data renamed as enif_priv_data.
  • enif_make_string got a third argument for character encoding.

A NIF library contains native implementation of some functions of an Erlang module. The native implemented functions (NIFs) are called like any other functions without any difference to the caller. Each NIF must also have an implementation in Erlang that will be invoked if the function is called before the NIF library has been successfully loaded. A typical such stub implementation is to throw an exception. But it can also be used as a fallback implementation if the NIF library is not implemented for some architecture.

A minimal example of a NIF library can look like this:

/* niftest.c */
#include "erl_nif.h"

static ERL_NIF_TERM hello(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
{
    return enif_make_string(env, "Hello world!", ERL_NIF_LATIN1);
}

static ErlNifFunc nif_funcs[] =
{
    {"hello", 0, hello}
};

ERL_NIF_INIT(niftest,nif_funcs,NULL,NULL,NULL,NULL)

and the Erlang module would have to look something like this:

-module(niftest).

-export([init/0, hello/0]).

init() ->
      erlang:load_nif("./niftest", 0).

hello() ->
      "NIF library not loaded".

and compile and test something like this (on Linux):

$> gcc -fPIC -shared -o niftest.so niftest.c -I $ERL_ROOT/usr/include/
$> erl

1> c(niftest).
{ok,niftest}
2> niftest:hello().
"NIF library not loaded"
3> niftest:init().
ok
4> niftest:hello().
"Hello world!"

A better solution for a real module is to take advantage of the new directive on_load to automatically load the NIF library when the module is loaded.

Note!

A NIF must be exported or used locally by the module (or both). An unused local stub function will be optimized away by the compiler causing loading of the NIF library to fail.

A loaded NIF library is tied to the Erlang module code version that loaded it. If the module is upgraded with a new version, the new Erlang code will have to load its own NIF library (or maybe choose not to). The new code version can however choose to load the exact same NIF library as the old code if it wants to. Sharing the same dynamic library will mean that static data defined by the library will be shared as well. To avoid unintentionally shared static data, each Erlang module code can keep its own private data. This private data can be set when the NIF library is loaded and then retrieved by calling enif_priv_data().

There is no way to explicitly unload a NIF library. A library will be automatically unloaded when the module code that it belongs to is purged by the code server. A NIF library will also be unloaded if it is replaced by another version of the library by a second call to erlang:load_nif/2 from the same module code.

FUNCTIONALITY

All functions that a NIF library needs to do with Erlang are performed through the NIF API functions. There are functions for the following functionality:

Read and write Erlang terms

Any Erlang terms can be passed to a NIF as function arguments and be returned as function return values. The terms are of C-type ERL_NIF_TERM and can only be read or written using API functions. Most functions to read the content of a term are prefixed enif_get_ and usually return true (or false) if the term was of the expected type (or not). The functions to write terms are all prefixed enif_make_ and usually return the created ERL_NIF_TERM. There are also some functions to query terms, like enif_is_atom, enif_is_identical and enif_compare.

Binaries

Terms of type binary are accessed with the help of the struct type ErlNifBinary that contains a pointer (data) to the raw binary data and the length (size) of the data in bytes. Both data and size are read-only and should only be written using calls to API functions. Instances of ErlNifBinary are however always allocated by the user (usually as local variables).

The raw data pointed to by data is only mutable after a call to enif_alloc_binary or enif_realloc_binary. All other functions that operates on a binary will leave the data as read-only. A mutable binary must in the end either be freed with enif_release_binary or made read-only by transferring it to an Erlang term with enif_make_binary. But it does not have do happen in the same NIF call. Read-only binaries does not have to be released.

Binaries are sequences of whole bytes. Bitstrings with an arbitrary bit length have no support yet.

Resource objects

The use of resource objects is a way to return pointers to native data structures from a NIF in a safe way. A resource object is just a block of memory allocated with enif_alloc_resource(). A handle ("safe pointer") to this memory block can then be returned to Erlang by the use of enif_make_resource(). The term returned by enif_make_resource is totally opaque in nature. It can be stored and passed between processses on the same node, but the only real end usage is to pass it back as argument to a NIF. The NIF can then do enif_get_resource() and get back a pointer to the memory block that is guaranteed to still be valid. A resource object will not be deallocated until the last handle term has been garbage collected by the VM and the resource has been released with enif_release_resource() (not necessarily in that order).

All resource objects are created as instances of some resource type. This makes resources from different modules to be distinguishable. A resource type is created by calling enif_open_resource_type() when a library is loaded. Objects of that resource type can then later be allocated and enif_get_resource verifies that the resource is of the expected type. A resource type can have a user supplied destructor function that is automatically called when resources of that type are released (by either the garbage collector or enif_release_resource). Resource types are uniquely identified by a supplied name string.

Resource types support upgrade in runtime by allowing a loaded NIF library to takeover an already existing resource type and thereby "inherit" all existing objects of that type. The destructor of the new library will thereafter be called for the inherited objects and the library with the old destructor function can be safely unloaded. Existing resource objects, of a module that is upgraded, must either be deleted or taken over by the new NIF library. The unloading of a library will be postponed as long as it exists resource objects with a destructor function in the library.

Here is a template example of how to create and return a resource object.

    ERL_NIF_TERM term;
    MyStruct* ptr = enif_alloc_resource(env, my_resource_type, sizeof(MyStruct));

    /* initialize struct ... */

    term = enif_make_resource(env, ptr);

    if (keep_a_reference_of_our_own) {
        /* store 'ptr' in static variable, private data or other resource object */
    }
    else {
        enif_release_resource(env, obj);
        /* resource now only owned by "Erlang" */
    }
    return term;
}
Threads and concurrency

A NIF is thread-safe without any explicit synchronization as long as it acts as a pure function and only reads the supplied arguments. As soon as you write towards a shared state either through static variables or enif_priv_data you need to supply your own explicit synchronization. Resource objects will also require synchronization if you treat them as mutable.

The library initialization callbacks load, reload and upgrade are all thread-safe even for shared state data.

Avoid doing lengthy work in NIF calls as that may degrade the responsiveness of the VM. NIFs are called directly by the same scheduler thread that executed the calling Erlang code. The calling scheduler will thus be blocked from doing any other work until the NIF returns.

INITIALIZATION

ERL_NIF_INIT(MODULE, ErlNifFunc funcs[], load, reload, upgrade, unload)

This is the magic macro to initialize a NIF library. It should be evaluated in global file scope.

MODULE is the name of the Erlang module as an identifier without string quotations. It will be stringified by the macro.

funcs is a static array of function descriptors for all the implemented NIFs in this library.

load, reload, upgrade and unload are pointers to functions. One of load, reload or upgrade will be called to initialize the library. unload is called to release the library. They are all described individually below.

int (*load)(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info)

load is called when the NIF library is loaded and there is no previously loaded library for this module.

*priv_data can be set to point to some private data that the library needs in able to keep a state between NIF calls. enif_priv_data() will return this pointer. *priv_data will be initialized to NULL when load is called.

load_info is the second argument to erlang:load_nif/2.

The library will fail to load if load returns anything other than 0. load can be NULL in case no initialization is needed.

int (*reload)(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info)

reload is called when the NIF library is loaded and there is already a previously loaded library for this module code.

Works the same as load. The only difference is that *priv_data already contains the value set by the previous call to load or reload.

The library will fail to load if reload returns anything other than 0 or if reload is NULL.

int (*upgrade)(ErlNifEnv* env, void** priv_data, void** old_priv_data, ERL_NIF_TERM load_info)

upgrade is called when the NIF library is loaded and there is no previously loaded library for this module code, BUT there is old code of this module with a loaded NIF library.

Works the same as load. The only difference is that *old_priv_data already contains the value set by the last call to load or reload for the old module code. *priv_data will be initialized to NULL when upgrade is called. It is allowed to write to both *priv_data and *old_priv_data.

The library will fail to load if upgrade returns anything other than 0 or if upgrade is NULL.

void (*unload)(ErlNifEnv* env, void* priv_data)

unload is called when the module code that the NIF library belongs to is purged as old. New code of the same module may or may not exist. Note that unload is not called for a replaced library as a consequence of reload.

DATA TYPES

ERL_NIF_TERM

Variables of type ERL_NIF_TERM can refer to any Erlang term. This is an opaque type and values of it can only by used either as arguments to API functions or as return values from NIFs. A variable of type ERL_NIF_TERM is only valid until the NIF call, where it was obtained, returns.

ErlNifEnv

ErlNifEnv contains information about the context in which a NIF call is made. This pointer should not be dereferenced in any way, but only passed on to API functions. An ErlNifEnv pointer is only valid until the function, where is what supplied as argument, returns. There is thus useless and dangerous to store ErlNifEnv pointers in between NIF calls.

ErlNifFunc

typedef struct {
    const char* name;
    unsigned arity;
    ERL_NIF_TERM (*fptr)(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]);
} ErlNifFunc;

Describes a NIF by its name, arity and implementation. fptr is a pointer to the function that implements the NIF. The argument argv of a NIF will contain the function arguments passed to the NIF and argc is the length of the array, i.e. the function arity. argv[N-1] will thus denote the Nth argument to the NIF. Note that the argc argument allows for the same C function to implement several Erlang functions with different arity (but same name probably).

ErlNifBinary

typedef struct {
    unsigned size;
    unsigned char* data;
} ErlNifBinary;

ErlNifBinary contains transient information about an inspected binary term. data is a pointer to a buffer of size bytes with the raw content of the binary.

ErlNifResourceType

Each instance of ErlNifResourceType represent a class of memory managed resource objects that can be garbage collected. Each resource type has a unique name and a destructor function that is called when objects of its type are released.

ErlNifResourceDtor

typedef void ErlNifResourceDtor(ErlNifEnv* env, void* obj);

The function prototype of a resource destructor function. A destructor function is not allowed to call any term-making functions.

ErlNifCharEncoding

typedef enum {
    ERL_NIF_LATIN1
}ErlNifCharEncoding;

The character encoding used in strings. The only supported encoding is currently ERL_NIF_LATIN1 for iso-latin-1 (8-bit ascii).

ErlNifSysInfo

Used by enif_system_info to return information about the runtime system. Contains currently the exact same content as ErlDrvSysInfo.

Functions

Allocate memory of size bytes. Return NULL if allocation failed.

Allocate a new binary of size of size bytes. Initialize the structure pointed to by bin to refer to the allocated binary. The binary must either be released by enif_release_binary() or ownership transferred to an Erlang term with enif_make_binary(). An allocated (and owned) ErlNifBinary can be kept between NIF calls.

Return false if allocation failed.

Allocate a memory managed resource object of type type and size size bytes.

Return an integer less than, equal to, or greater than zero if lhs is found, respectively, to be less than, equal, or greater than rhs. Corresponds to the Erlang operators ==, /=, =<, <, >= and > (but not =:= or =/=).

Same as erl_drv_cond_broadcast().

Same as erl_drv_cond_create().

Same as erl_drv_cond_destroy().

Same as erl_drv_cond_signal().

Same as erl_drv_cond_wait().

Same as erl_drv_equal_tids().

Free memory allocated by enif_alloc.

Write a null-terminated string, in the buffer pointed to by buf of size size, consisting of the string representation of the atom term. Return the number of bytes written (including terminating null character) or 0 if term is not an atom with maximum length of size-1.

Set *dp to the floating point value of term or return false if term is not a float.

Set *ip to the integer value of term or return false if term is not an integer or is outside the bounds of type int

Set *head and *tail from list or return false if list is not a non-empty list.

Set *ip to the long integer value of term or return false if term is not an integer or is outside the bounds of type long int.

Set *objp to point to the resource object referred to by term. The pointer is valid until the calling NIF returns and should not be released.

Return false if term is not a handle to a resource object of type type.

Write a null-terminated string, in the buffer pointed to by buf with size size, consisting of the characters in the string list. The characters are written using encoding encode. Return the number of bytes written (including terminating null character), or -size if the string was truncated due to buffer space, or 0 if list is not a string that can be encoded with encode or if size was less than 1. The written string is always null-terminated unless buffer size is less than 1.

If term is a tuple, set *array to point to an array containing the elements of the tuple and set *arity to the number of elements. Note that the array is read-only and (*array)[N-1] will be the Nth element of the tuple. *array is undefined if the arity of the tuple is zero.

Return false if term is not a tuple.

Set *ip to the unsigned integer value of term or return false if term is not an unsigned integer or is outside the bounds of type unsigned int

Set *ip to the unsigned long integer value of term or return false if term is not an unsigned integer or is outside the bounds of type unsigned long

Initialize the structure pointed to by bin with information about the binary term bin_term. Return false if bin_term is not a binary.

Initialize the structure pointed to by bin with one continuous buffer with the same byte content as iolist. As with inspect_binary, the data pointed to by bin is transient and does not need to be released. Return false if iolist is not an iolist.

Return true if term is an atom.

Return true if term is a binary

Return true if term is an empty list.

Return true if term is a fun.

Return true if the two terms are identical. Corresponds to the Erlang operators =:= and =/=.

Return true if term is a pid.

Return true if term is a port.

Return true if term is a reference.

Create an atom term from the C-string name. Unlike other terms, atom terms may be saved and used between NIF calls.

Make a badarg exception to be returned from a NIF.

Make a binary term from bin. Any ownership of the binary data will be transferred to the created term and bin should be considered read-only for the rest of the NIF call and then as released.

Create an floating-point term from a double.

Try to create the term of an already existing atom from the C-string name. If the atom already exist store the term in *atom and return true, otherwise return false.

Create an integer term.

Create an ordinary list term of length cnt. Expects cnt number of arguments (after cnt) of type ERL_NIF_TERM as the elements of the list. An empty list is returned if cnt is 0.

Create an ordinary list term with length indicated by the function name. Prefer these functions (macros) over the variadic enif_make_list to get compile time error if the number of arguments does not match.

Create a list cell [head | tail].

Create an ordinary list containing the elements of array arr of length cnt. An empty list is returned if cnt is 0.

Create an integer term from a long int.

Create a reference like erlang:make_ref/0.

Create an opaque handle to a memory managed resource object obtained by enif_alloc_resource. No ownership transfer is done, the resource object still needs to be released by enif_release_resource.

Note that the only defined behaviour when using of a resource term in an Erlang program is to store it and send it between processes on the same node. Other operations such as matching or term_to_binary will have unpredictable (but harmless) results.

Create a list containing the characters of the null-terminated string string with encoding encoding.

Make a subbinary of binary bin_term, starting at zero-based position pos with a length of size bytes. bin_term must be a binary or bitstring and pos+size must be less or equal to the number of whole bytes in bin_term.

Create a tuple term of arity cnt. Expects cnt number of arguments (after cnt) of type ERL_NIF_TERM as the elements of the tuple.

Create a tuple term with length indicated by the function name. Prefer these functions (macros) over the variadic enif_make_tuple to get compile time error if the number of arguments does not match.

Create a tuple containing the elements of array arr of length cnt.

Create an integer term from an unsigned int.

Create an integer term from an unsigned long int.

Same as erl_drv_mutex_create().

Same as erl_drv_mutex_destroy().

Same as erl_drv_mutex_lock().

Same as erl_drv_mutex_trylock().

Same as erl_drv_mutex_unlock().

Create or takeover a resource type identified by the string name and give it the destructor function pointed to by dtor. Argument flags can have the following values:

ERL_NIF_RT_CREATE
Create a new resource type that does not already exist.
ERL_NIF_RT_TAKEOVER
Open an existing resource type and take over ownership of all its instances. The supplied destructor dtor will be called both for existing instances as well as new instances not yet created by the calling NIF library.

The two flag values can be combined with bitwise-or. To avoid unintentionally name clashes a good practice is to include the module name as part of the type name. The dtor may be NULL in case no destructor is needed.

On success, return a pointer to the resource type and *tried will be set to either ERL_NIF_RT_CREATE or ERL_NIF_RT_TAKEOVER to indicate what was actually done. On failure, return NULL and set *tried to flags. It is allowed to set tried to NULL.

Note that enif_open_resource_type is only allowed to be called in the three callbacks load, reload and upgrade.

Return the pointer to the private data that was set by load, reload or upgrade.

Was previously named enif_get_data.

Change the size of a binary bin. The source binary may be read-only, in which case it will be left untouched and a mutable copy is allocated and assigned to *bin.

Release a binary obtained from enif_alloc_binary.

Release a resource objects obtained from enif_alloc_resource. The object may still be alive if it is referred to by Erlang terms. Each call to enif_release_resource must correspond to a previous call to enif_alloc_resource. References made by enif_make_resource can only be released by the garbage collector.

Same as erl_drv_rwlock_create().

Same as erl_drv_rwlock_destroy().

Same as erl_drv_rwlock_rlock().

Same as erl_drv_rwlock_runlock().

Same as erl_drv_rwlock_rwlock().

Same as erl_drv_rwlock_rwunlock().

Same as erl_drv_rwlock_tryrlock().

Same as erl_drv_rwlock_tryrwlock().

Get the byte size of a resource object obj obtained by enif_alloc_resource.

Same as driver_system_info().

Same as erl_drv_thread_create().

Same as erl_drv_thread_exit().

Same as erl_drv_thread_join ().

Same as erl_drv_thread_opts_create().

Same as erl_drv_thread_opts_destroy().

Same as erl_drv_thread_self().

Same as erl_drv_tsd_key_create().

Same as erl_drv_tsd_key_destroy().

Same as erl_drv_tsd_get().

Same as erl_drv_tsd_set().

SEE ALSO

load_nif(3)