erl_nif

API functions for an Erlang NIF library

Warning!

The NIF concept was introduced in R13B03 as an EXPERIMENTAL feature. The interfaces may be changed in any way in coming releases. The plan is however to lift the experimental label and maintain interface backward compatibility from R14B.

Incompatible changes in R14A:

Environment argument removed for enif_alloc, enif_realloc, enif_free, enif_alloc_binary, enif_realloc_binary, enif_release_binary, enif_alloc_resource, enif_release_resource, enif_is_identical and enif_compare.
Character encoding argument added to enif_get_atom and enif_make_existing_atom.
Module argument added to enif_open_resource_type while changing name spaces of resource types from global to module local.

Incompatible changes in R13B04:

The function prototypes of the NIFs have changed to expect argc and argv arguments. The arity of a NIF is by that no longer limited to 3.
enif_get_data renamed as enif_priv_data.
enif_make_string got a third argument for character encoding.

A NIF library contains native implementation of some functions of an Erlang module. The native implemented functions (NIFs) are called like any other functions without any difference to the caller. Each NIF must also have an implementation in Erlang that will be invoked if the function is called before the NIF library has been successfully loaded. A typical such stub implementation is to throw an exception. But it can also be used as a fallback implementation if the NIF library is not implemented for some architecture.

A minimal example of a NIF library can look like this:

/* niftest.c */
#include "erl_nif.h"

static ERL_NIF_TERM hello(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
{
    return enif_make_string(env, "Hello world!", ERL_NIF_LATIN1);
}

static ErlNifFunc nif_funcs[] =
{
    {"hello", 0, hello}
};

ERL_NIF_INIT(niftest,nif_funcs,NULL,NULL,NULL,NULL)

and the Erlang module would have to look something like this:

-module(niftest).

-export([init/0, hello/0]).

init() ->
      erlang:load_nif("./niftest", 0).

hello() ->
      "NIF library not loaded".

and compile and test something like this (on Linux):

$> gcc -fPIC -shared -o niftest.so niftest.c -I $ERL_ROOT/usr/include/
$> erl

1> c(niftest).
{ok,niftest}
2> niftest:hello().
"NIF library not loaded"
3> niftest:init().
ok
4> niftest:hello().
"Hello world!"

A better solution for a real module is to take advantage of the new directive on_load to automatically load the NIF library when the module is loaded.

Note!

A NIF does not have to be exported, it can be local to the module. Note however that unused local stub functions will be optimized away by the compiler causing loading of the NIF library to fail.

A loaded NIF library is tied to the Erlang module code version that loaded it. If the module is upgraded with a new version, the new Erlang code will have to load its own NIF library (or maybe choose not to). The new code version can however choose to load the exact same NIF library as the old code if it wants to. Sharing the same dynamic library will mean that static data defined by the library will be shared as well. To avoid unintentionally shared static data, each Erlang module code can keep its own private data. This private data can be set when the NIF library is loaded and then retrieved by calling enif_priv_data.

There is no way to explicitly unload a NIF library. A library will be automatically unloaded when the module code that it belongs to is purged by the code server. A NIF library will also be unloaded if it is replaced by another version of the library by a second call to erlang:load_nif/2 from the same module code.

FUNCTIONALITY

All functions that a NIF library needs to do with Erlang are performed through the NIF API functions. There are functions for the following functionality:

Read and write Erlang terms

Any Erlang terms can be passed to a NIF as function arguments and be returned as function return values. The terms are of C-type ERL_NIF_TERM and can only be read or written using API functions. Most functions to read the content of a term are prefixed enif_get_ and usually return true (or false) if the term was of the expected type (or not). The functions to write terms are all prefixed enif_make_ and usually return the created ERL_NIF_TERM. There are also some functions to query terms, like enif_is_atom, enif_is_identical and enif_compare.

All terms of type ERL_NIF_TERM belong to an environment of type ErlNifEnv. The lifetime of a term is controlled by the lifetime of its environment object. All API functions that read or write terms has the environment, that the term belongs to, as the first function argument.

Binaries

Terms of type binary are accessed with the help of the struct type ErlNifBinary that contains a pointer (data) to the raw binary data and the length (size) of the data in bytes. Both data and size are read-only and should only be written using calls to API functions. Instances of ErlNifBinary are however always allocated by the user (usually as local variables).

The raw data pointed to by data is only mutable after a call to enif_alloc_binary or enif_realloc_binary. All other functions that operates on a binary will leave the data as read-only. A mutable binary must in the end either be freed with enif_release_binary or made read-only by transferring it to an Erlang term with enif_make_binary. But it does not have to happen in the same NIF call. Read-only binaries do not have to be released.

enif_make_new_binary can be used as a shortcut to allocate and return a binary in the same NIF call.

Binaries are sequences of whole bytes. Bitstrings with an arbitrary bit length have no support yet.

Resource objects

The use of resource objects is a way to return pointers to native data structures from a NIF in a safe way. A resource object is just a block of memory allocated with enif_alloc_resource. A handle ("safe pointer") to this memory block can then be returned to Erlang by the use of enif_make_resource. The term returned by enif_make_resource is totally opaque in nature. It can be stored and passed between processses on the same node, but the only real end usage is to pass it back as argument to a NIF. The NIF can then do enif_get_resource and get back a pointer to the memory block that is guaranteed to still be valid. A resource object will not be deallocated until the last handle term has been garbage collected by the VM and the resource has been released with enif_release_resource (not necessarily in that order).

All resource objects are created as instances of some resource type. This makes resources from different modules to be distinguishable. A resource type is created by calling enif_open_resource_type when a library is loaded. Objects of that resource type can then later be allocated and enif_get_resource verifies that the resource is of the expected type. A resource type can have a user supplied destructor function that is automatically called when resources of that type are released (by either the garbage collector or enif_release_resource). Resource types are uniquely identified by a supplied name string and the name of the implementing module.

Resource types support upgrade in runtime by allowing a loaded NIF library to takeover an already existing resource type and thereby "inherit" all existing objects of that type. The destructor of the new library will thereafter be called for the inherited objects and the library with the old destructor function can be safely unloaded. Existing resource objects, of a module that is upgraded, must either be deleted or taken over by the new NIF library. The unloading of a library will be postponed as long as there exist resource objects with a destructor function in the library.

Here is a template example of how to create and return a resource object.

    ERL_NIF_TERM term;
    MyStruct* ptr = enif_alloc_resource(my_resource_type, sizeof(MyStruct));

    /* initialize struct ... */

    term = enif_make_resource(env, ptr);

    if (keep_a_reference_of_our_own) {
        /* store 'ptr' in static variable, private data or other resource object */
    }
    else {
        enif_release_resource(obj);
        /* resource now only owned by "Erlang" */
    }
    return term;
}

Another usage of resource objects is to create binary terms with user defined memory management. enif_make_resource_binary will create a binary term that is connected to a resource object. The destructor of the resource will be called when the binary is garbage collected, at which time the binary data can be released. An example of this can be a binary term consisting of data from a mmap'ed file. The destructor can then do munmap to release the memory region.

Threads and concurrency

A NIF is thread-safe without any explicit synchronization as long as it acts as a pure function and only reads the supplied arguments. As soon as you write towards a shared state either through static variables or enif_priv_data you need to supply your own explicit synchronization. This includes terms in process independent environments that are shared between threads. Resource objects will also require synchronization if you treat them as mutable.

The library initialization callbacks load, reload and upgrade are all thread-safe even for shared state data.

Avoid doing lengthy work in NIF calls as that may degrade the responsiveness of the VM. NIFs are called directly by the same scheduler thread that executed the calling Erlang code. The calling scheduler will thus be blocked from doing any other work until the NIF returns.

INITIALIZATION

ERL_NIF_INIT(MODULE, ErlNifFunc funcs[], load, reload, upgrade, unload)

This is the magic macro to initialize a NIF library. It should be evaluated in global file scope.

MODULE is the name of the Erlang module as an identifier without string quotations. It will be stringified by the macro.

funcs is a static array of function descriptors for all the implemented NIFs in this library.

load, reload, upgrade and unload are pointers to functions. One of load, reload or upgrade will be called to initialize the library. unload is called to release the library. They are all described individually below.

int (*load)(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info)

load is called when the NIF library is loaded and there is no previously loaded library for this module.

*priv_data can be set to point to some private data that the library needs in order to keep a state between NIF calls. enif_priv_data will return this pointer. *priv_data will be initialized to NULL when load is called.

load_info is the second argument to erlang:load_nif/2.

The library will fail to load if load returns anything other than 0. load can be NULL in case no initialization is needed.

int (*reload)(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info)

reload is called when the NIF library is loaded and there is already a previously loaded library for this module code.

Works the same as load. The only difference is that *priv_data already contains the value set by the previous call to load or reload.

The library will fail to load if reload returns anything other than 0 or if reload is NULL.

int (*upgrade)(ErlNifEnv* env, void** priv_data, void** old_priv_data, ERL_NIF_TERM load_info)

upgrade is called when the NIF library is loaded and there is no previously loaded library for this module code, BUT there is old code of this module with a loaded NIF library.

Works the same as load. The only difference is that *old_priv_data already contains the value set by the last call to load or reload for the old module code. *priv_data will be initialized to NULL when upgrade is called. It is allowed to write to both *priv_data and *old_priv_data.

The library will fail to load if upgrade returns anything other than 0 or if upgrade is NULL.

void (*unload)(ErlNifEnv* env, void* priv_data)

unload is called when the module code that the NIF library belongs to is purged as old. New code of the same module may or may not exist. Note that unload is not called for a replaced library as a consequence of reload.

DATA TYPES

ERL_NIF_TERM

Variables of type ERL_NIF_TERM can refer to any Erlang term. This is an opaque type and values of it can only by used either as arguments to API functions or as return values from NIFs. All ERL_NIF_TERM's belong to an environment (ErlNifEnv). A term can not be destructed individually, it is valid until its environment is destructed.

ErlNifEnv

ErlNifEnv represents an environment that can host Erlang terms. All terms in an environment are valid as long as the environment is valid. ErlNifEnv is an opaque type and pointers to it can only be passed on to API functions. There are two types of environments; process bound and process independent.

A process bound environment is passed as the first argument to all NIFs. All function arguments passed to a NIF will belong to that environment. The return value from a NIF must also be a term belonging to the same environment. In addition a process bound environment contains transient information about the calling Erlang process. The environment is only valid in the thread where it was supplied as argument until the NIF returns. It is thus useless and dangerous to store pointers to process bound environments between NIF calls.

A process independent environment is created by calling enif_alloc_env. It can be used to store terms beteen NIF calls and to send terms with enif_send. A process independent environment with all its terms is valid until you explicitly invalidates it with enif_free_env or enif_send.

All elements of a list/tuple must belong to the same environment as the list/tuple itself. Terms can be copied between environments with enif_make_copy.

ErlNifFunc

typedef struct {
    const char* name;
    unsigned arity;
    ERL_NIF_TERM (*fptr)(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]);
} ErlNifFunc;

Describes a NIF by its name, arity and implementation. fptr is a pointer to the function that implements the NIF. The argument argv of a NIF will contain the function arguments passed to the NIF and argc is the length of the array, i.e. the function arity. argv[N-1] will thus denote the Nth argument to the NIF. Note that the argc argument allows for the same C function to implement several Erlang functions with different arity (but same name probably).

ErlNifBinary

typedef struct {
    unsigned size;
    unsigned char* data;
} ErlNifBinary;

ErlNifBinary contains transient information about an inspected binary term. data is a pointer to a buffer of size bytes with the raw content of the binary.

Note that ErlNifBinary is a semi-opaque type and you are only allowed to read fields size and data.

ErlNifPid

ErlNifPid is a process identifier (pid). In contrast to pid terms (instances of ERL_NIF_TERM), ErlNifPid's are self contained and not bound to any environment. ErlNifPid is an opaque type.

ErlNifResourceType

Each instance of ErlNifResourceType represent a class of memory managed resource objects that can be garbage collected. Each resource type has a unique name and a destructor function that is called when objects of its type are released.

ErlNifResourceDtor

typedef void ErlNifResourceDtor(ErlNifEnv* env, void* obj);

The function prototype of a resource destructor function. A destructor function is not allowed to call any term-making functions.

ErlNifCharEncoding

typedef enum {
    ERL_NIF_LATIN1
}ErlNifCharEncoding;

The character encoding used in strings and atoms. The only supported encoding is currently ERL_NIF_LATIN1 for iso-latin-1 (8-bit ascii).

ErlNifSysInfo

Used by enif_system_info to return information about the runtime system. Contains currently the exact same content as ErlDrvSysInfo.

Functions

Allocate memory of size bytes. Return NULL if allocation failed.

Allocate a new binary of size size bytes. Initialize the structure pointed to by bin to refer to the allocated binary. The binary must either be released by enif_release_binary or ownership transferred to an Erlang term with enif_make_binary. An allocated (and owned) ErlNifBinary can be kept between NIF calls.

Return true on success or false if allocation failed.

Allocate a new process independent environment. The environment can be used to hold terms that is not bound to any process. Such terms can later be copied to a process environment with enif_make_copy or be sent to a process as a message with enif_send.

Return pointer to the new environment.

Allocate a memory managed resource object of type type and size size bytes.

Free all terms in an environment and clear it for reuse. The environment must have been allocated with enif_alloc_env.

Return an integer less than, equal to, or greater than zero if lhs is found, respectively, to be less than, equal, or greater than rhs. Corresponds to the Erlang operators ==, /=, =<, <, >= and > (but not =:= or =/=).

Same as erl_drv_cond_broadcast.

Same as erl_drv_cond_create.

Same as erl_drv_cond_destroy.

Same as erl_drv_cond_signal.

Same as erl_drv_cond_wait.

Same as erl_drv_equal_tids.

Free memory allocated by enif_alloc.

Free an environment allocated with enif_alloc_env. All terms created in the environment will be freed as well.

Write a null-terminated string, in the buffer pointed to by buf of size size, consisting of the string representation of the atom term with encoding encode. Return the number of bytes written (including terminating null character) or 0 if term is not an atom with maximum length of size-1.

Set *len to the length (number of bytes excluding terminating null character) of the atom term with encoding encode. Return true on success or false if term is not an atom.

Set *dp to the floating point value of term. Return true on success or false if term is not a float.

Set *ip to the integer value of term. Return true on success or false if term is not an integer or is outside the bounds of type int

If term is the pid of a node local process, initialize the pid variable *pid from it and return true. Otherwise return false. No check if the process is alive is done.

Set *head and *tail from list and return true, or return false if list is not a non-empty list.

Set *len to the length of list term and return true, or return false if term is not a list.

Set *ip to the long integer value of term and return true, or return false if term is not an integer or is outside the bounds of type long int.

Set *objp to point to the resource object referred to by term.

Return true on success or false if term is not a handle to a resource object of type type.

Write a null-terminated string, in the buffer pointed to by buf with size size, consisting of the characters in the string list. The characters are written using encoding encode. Return the number of bytes written (including terminating null character), or -size if the string was truncated due to buffer space, or 0 if list is not a string that can be encoded with encode or if size was less than 1. The written string is always null-terminated unless buffer size is less than 1.

If term is a tuple, set *array to point to an array containing the elements of the tuple and set *arity to the number of elements. Note that the array is read-only and (*array)[N-1] will be the Nth element of the tuple. *array is undefined if the arity of the tuple is zero.

Return true on success or false if term is not a tuple.

Set *ip to the unsigned integer value of term and return true, or return false if term is not an unsigned integer or is outside the bounds of type unsigned int.

Set *ip to the unsigned long integer value of term and return true, or return false if term is not an unsigned integer or is outside the bounds of type unsigned long.

Initialize the structure pointed to by bin with information about the binary term bin_term. Return true on success or false if bin_term is not a binary.

Initialize the structure pointed to by bin with one continuous buffer with the same byte content as iolist. As with inspect_binary, the data pointed to by bin is transient and does not need to be released. Return true on success or false if iolist is not an iolist.

Return true if term is an atom.

Return true if term is a binary

Return true if term is an empty list.

Return true if term is a fun.

Return true if the two terms are identical. Corresponds to the Erlang operators =:= and =/=.

Return true if term is a pid.

Return true if term is a port.

Return true if term is a reference.

Return true if term is a tuple.

Return true if term is a list.

Add a reference to resource object obj obtained from enif_alloc_resource. Each call to enif_keep_resource for an object must be balanced by a call to enif_release_resource before the object will be destructed.

Create an atom term from the null-terminated C-string name with iso-latin-1 encoding.

Create an atom term from the string name with length len. Null-characters are treated as any other characters.

Make a badarg exception to be returned from a NIF.

Make a binary term from bin. Any ownership of the binary data will be transferred to the created term and bin should be considered read-only for the rest of the NIF call and then as released.

Make a copy of term src_term. The copy will be created in environment dst_env. The source term may be located in any environment.

Create a floating-point term from a double.

Try to create the term of an already existing atom from the null-terminated C-string name with encoding encode. If the atom already exists store the term in *atom and return true, otherwise return false.

Try to create the term of an already existing atom from the string name with length len and encoding encode. Null-characters are treated as any other characters. If the atom already exists store the term in *atom and return true, otherwise return false.

Create an integer term.

Create an ordinary list term of length cnt. Expects cnt number of arguments (after cnt) of type ERL_NIF_TERM as the elements of the list. An empty list is returned if cnt is 0.

Create an ordinary list term with length indicated by the function name. Prefer these functions (macros) over the variadic enif_make_list to get a compile time error if the number of arguments does not match.

Create a list cell [head | tail].

Create an ordinary list containing the elements of array arr of length cnt. An empty list is returned if cnt is 0.

Create an integer term from a long int.

Allocate a binary of size size bytes and create an owning term. The binary data is mutable until the calling NIF returns. This is a quick way to create a new binary without having to use ErlNifBinary. The drawbacks are that the binary can not be kept between NIF calls and it can not be reallocated.

Return a pointer to the raw binary data and set *termp to the binary term.

Make a pid term from *pid.

Create a reference like erlang:make_ref/0.

Create an opaque handle to a memory managed resource object obtained by enif_alloc_resource. No ownership transfer is done, the resource object still needs to be released by enif_release_resource.

Note that the only defined behaviour of using a resource term in an Erlang program is to store it and send it between processes on the same node. Other operations such as matching or term_to_binary will have unpredictable (but harmless) results.

Create a binary term that is memory managed by a resource object obj obtained by enif_alloc_resource. The returned binary term will consist of size bytes pointed to by data. This raw binary data must be kept readable and unchanged until the destructor of the resource is called. The binary data may be stored external to the resource object in which case it is the responsibility of the destructor to release the data.

Several binary terms may be managed by the same resource object. The destructor will not be called until the last binary is garbage collected. This can be useful as a way to return different parts of a larger binary buffer.

As with enif_make_resource, no ownership transfer is done. The resource still needs to be released with enif_release_resource.

Initialize the pid variable *pid to represent the calling process. Return pid.

Create a list containing the characters of the null-terminated string string with encoding encoding.

Create a list containing the characters of the string string with length len and encoding encoding. Null-characters are treated as any other characters.

Make a subbinary of binary bin_term, starting at zero-based position pos with a length of size bytes. bin_term must be a binary or bitstring and pos+size must be less or equal to the number of whole bytes in bin_term.

Create a tuple term of arity cnt. Expects cnt number of arguments (after cnt) of type ERL_NIF_TERM as the elements of the tuple.

Create a tuple term with length indicated by the function name. Prefer these functions (macros) over the variadic enif_make_tuple to get a compile time error if the number of arguments does not match.

Create a tuple containing the elements of array arr of length cnt.

Create an integer term from an unsigned int.

Create an integer term from an unsigned long int.

Same as erl_drv_mutex_create.

Same as erl_drv_mutex_destroy.

Same as erl_drv_mutex_lock.

Same as erl_drv_mutex_trylock.

Same as erl_drv_mutex_unlock.

Create or takeover a resource type identified by the string name and give it the destructor function pointed to by dtor. Argument flags can have the following values:

ERL_NIF_RT_CREATE: Create a new resource type that does not already exist.
ERL_NIF_RT_TAKEOVER: Open an existing resource type and take over ownership of all its instances. The supplied destructor dtor will be called both for existing instances as well as new instances not yet created by the calling NIF library.

The two flag values can be combined with bitwise-or. The name of the resource type is local to the calling module. Argument module_str is not (yet) used and must be NULL. The dtor may be NULL in case no destructor is needed.

On success, return a pointer to the resource type and *tried will be set to either ERL_NIF_RT_CREATE or ERL_NIF_RT_TAKEOVER to indicate what was actually done. On failure, return NULL and set *tried to flags. It is allowed to set tried to NULL.

Note that enif_open_resource_type is only allowed to be called in the three callbacks load, reload and upgrade.

Return the pointer to the private data that was set by load, reload or upgrade.

Was previously named enif_get_data.

Change the size of a binary bin. The source binary may be read-only, in which case it will be left untouched and a mutable copy is allocated and assigned to *bin.

Release a binary obtained from enif_alloc_binary.

Remove a reference to resource object objobtained from enif_alloc_resource. The resource object will be destructed when the last reference is removed. Each call to enif_release_resource must correspond to a previous call to enif_alloc_resource or enif_keep_resource. References made by enif_make_resource can only be removed by the garbage collector.

Same as erl_drv_rwlock_create.

Same as erl_drv_rwlock_destroy.

Same as erl_drv_rwlock_rlock.

Same as erl_drv_rwlock_runlock.

Same as erl_drv_rwlock_rwlock.

Same as erl_drv_rwlock_rwunlock.

Same as erl_drv_rwlock_tryrlock.

Same as erl_drv_rwlock_tryrwlock.

Send a message to a process.

env: The environment of the calling process. Must be NULL if and only if calling from a created thread.
*to_pid: The pid of the receiving process. The pid should refer to a process on the local node.
msg_env: The environment of the message term. Must be a process independent environment allocated with enif_alloc_env.
msg: The message term to send.

Return true on success, or false if *to_pid does not refer to an alive local process.

The message environment msg_env with all its terms (including msg) will be invalidated by a successful call to enif_send. The environment should either be freed with enif_free_env of cleared for reuse with enif_clear_env.

This function is only thread-safe when the emulator with SMP support is used. It can only be used in a non-SMP emulator from a NIF-calling thread.

Get the byte size of a resource object obj obtained by enif_alloc_resource.

Same as driver_system_info.

Same as erl_drv_thread_create.

Same as erl_drv_thread_exit.

Same as erl_drv_thread_join .

Same as erl_drv_thread_opts_create.

Same as erl_drv_thread_opts_destroy.

Same as erl_drv_thread_self.

Same as erl_drv_tsd_key_create.

Same as erl_drv_tsd_key_destroy.

Same as erl_drv_tsd_get.