Ruby Memory Allocation

During a recent discussion at work, my ever so sharp coworker James Tucker noted the significant void between user code and low level optimizations.

Micro benchmarks and rubyspec compliance are frequently referenced to compare implementations and quite often cited in random rants as well.What happens at that intermediate, libc, level ?

In this post we’ll explore the ISO C memory allocation / deallocation functions as invoked by the following interpreters :

* ruby-1.8.7-p72 (./configure)
* ruby-1.9.1-p129 (./configure)
* jruby-1.3.1 (binary distribution)

Requirements

* A DTrace enabled OS (OS X Leopard, Solaris etc.)
* One or more interpreters installed
* A local checkout of extlib

I’ve opted to NOT test with the latest 1.9.2 preview – see why

The DTrace PID provider

A set of C level probes is spawned for the lifetime of a given command or existing process.When the process dies, the probe’s is unset.You don’t have to recompile or relink – it just works.See the Sun wiki for further information.

This is very different from static probes that ships with MySQL 6 or a previously released patched tree by Joyent.

Memory Allocation

Regardless of the underlying GC implementation, there’s three ISO C functions that handle memory allocation.

* malloc allocates X bytes of memory
* calloc allocates memory for X objects of Y size each
* realloc shrinks or expands the size of a previously allocated memory region

Each allocation function returns a pointer to the allocated (or reallocated) memory region which can be released with the free function call.

Of note is that most implementations never release memory from the heap back to the underlying OS – it’s assigned to a pool for reallocation to reduce system call (sbrk) overhead of the current process.

Toolbox

Ruby 1.8.7

I happen to use this as my stock 1.8 series interpreter – 1.8.6 should yield similar output.

Ruby 1.9.1

JRuby 1.3.1

Again, I’ve noticed a 1.4 version flying around, but happened to have 1.3.1 installed.

Passing Thoughts

Note the liberal memory allocation overheads on the 1.8 series, but an otherwise overall similar distribution pattern to 1.9.I can’t comment much on the JVM case due to limited Java experience.Anyone care to chime in ?

This is an exercise to test the waters for a potential audience for these experiments.I have a few contexts lined up, with a specific MRI focus, but can get into the bowels of most things.Suggestions and thoughts welcome.