Rails, Ruby & Prototype ramblings - Home

Exception Notification and Edge

2008-02-04T02:20:00Z

Changeset 8669 breaks the exception_notification plugin ... raising Unprocessed view path found in <somewhere>.

Drop the following in config/initializers/exception_notification.rb ...

ExceptionNotifier.exception_recipients = %w(you@yourdomain.com)
ExceptionNotifier.class_eval do 
  remove_method :template_root 
  ExceptionNotifier.template_root = "#{RAILS_ROOT}/vendor/plugins/exception_notification/lib/../views" 
end

Edge callback refactorings && attachment_fu

2008-01-19T12:06:00Z

Edge changeset 8664 introduces ActiveSupport::Callbacks.

This currently breaks attachment_fu's callback internals and may affect other plugins as well ...

module Technoweenie
  module AttachmentFu

    module InstanceMethods

      def self.included( base )
        base.define_callbacks *[:after_resize, :after_attachment_saved, :before_thumbnail_saved]
      end  

      def callback_with_args(method, arg = self)
         notify(method)

          result = run_callbacks(method, { :object => arg }) { |result, object| result == false }

          if result != false && respond_to_without_attributes?(method)
            result = send(method)
          end

          return result
      end      

      def run_callbacks(kind, options = {}, &block)
        options.reverse_merge!( :object => self )
        ::ActiveSupport::Callbacks::Callback.run(self.class.send("#{kind}_callback_chain"), options[:object], options, &block)
      end      
    end
  end
end

Hope the above monkey patch relieves a bout of intense swearing.

Get your Caffeine on

2007-06-04T02:26:00Z

I recently stumbled upon Adocca plugins, which is a suite of plugins / extensions extracted from the development of trig.com.

This includes a nifty Ruby extension, Caffeine, built on libmemcache, the C API for Memcached.The API is almost 100% compatible with memcache-client, with the following exceptions :

No get_multi ( may or may not be a big deal for you )
Multithreaded by default, only accepts a single :namespace as options Hash
MemCache#servers is write only and you can't overwrite previously defined servers
MemCache#get accepts either a String or Array, the latter which forces a namespace lookup (more on this later)

I'll continue with a quick rundown ...

Example installation ( OS X )
Requires a properly configured Memcached environment

Libmemcache installation

Grab it here, extract and ....


libmemcache-1.4.0.rc2 lourens$ ./configure && make && make install

Libuuid installation

Download the e2fsprogs package, extract and ...


e2fsprogs-1.39 lourens$ ./configure
e2fsprogs-1.39 lourens$  cd lib/uuid
e2fsprogs-1.39 lourens$ make && sudo make install

We don't need the whole package, just libuuid.

Caffeine installation


lourens$ sudo gem install caffeine
Building native extensions.  This could take a while...
caffeine.c: In function 'key_from_segments':
caffeine.c:712: warning: incompatible implicit declaration of built-in function 'mempcpy'
caffeine.c: In function 'md5_digest':
caffeine.c:872: warning: pointer targets in passing argument 1 of 'MD5' differ in signedness
/usr/bin/ld: warning multiple definitions of symbol _setregid
/opt/local/lib/libruby.dylib(process.o) definition of _setregid
/usr/lib/gcc/i686-apple-darwin8/4.0.1/../../../libpthread.dylib(setregid.So) definition of _setregid
/usr/bin/ld: warning multiple definitions of symbol _setreuid
/opt/local/lib/libruby.dylib(process.o) definition of _setreuid
/usr/lib/gcc/i686-apple-darwin8/4.0.1/../../../libpthread.dylib(setreuid.So) definition of _setreuid
ruby extconf.rb install caffeine
checking for mc_new() in -lmemcache... yes
checking for uuid_generate() in -luuid... yes
checking for main() in -lssl... yes
creating Makefile

make
gcc -I. -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.9.1 -I. -O -pipe -I/opt/local/include -fno-common -ggdb -O0 -Df  -c caffeine.c
cc -dynamic -bundle -undefined suppress -flat_namespace -L/opt/local/lib   -L"/opt/local/lib" -o caffeine.bundle caffeine.o  -lruby -lssl -luuid -lmemcache  -lpthread -ldl -lobjc -lspread 

make install
/usr/bin/install -c -m 0755 caffeine.bundle /opt/local/lib/ruby/gems/1.8/gems/caffeine-0.0.2/lib

make clean
Successfully installed caffeine-0.0.2

Namespaces

The interesting aspect about this client lib is the elegant namespace support, not to be confused with your connection namespace.One could have keys like the following (pseudo code) :

bookings-development:Account:1:Addresses:2
bookings-development:Account:20:Addresses:12

Very simple to sweep a whole set of related keys eg. to expire Account information, we just unlink the Account namespace.Actual keys resemble this ...

bookings-development:7f722ed3a969554d010e62510217cc

... courtesy of libuuid to avoid namespace / key trashing.

Performance?

I didn't gain all that much by using Caffeine in my dev / staging environment, but mileage varies, I like to keep good karma and thus won't open a 'scaling / performance' can of worms.

Integration

The following svn diff applied to cache_fu revision 240 should have you up and running without hickups.


lourens$ svn diff vendor/plugins/cache_fu
Index: vendor/plugins/cache_fu/lib/acts_as_cached/config.rb
===================================================================
--- vendor/plugins/cache_fu/lib/acts_as_cached/config.rb        (revision 2337)
+++ vendor/plugins/cache_fu/lib/acts_as_cached/config.rb        (working copy)
@@ -42,9 +42,16 @@

     def setup_memcache(config)
       config[:namespace] << "-#{RAILS_ENV}"
-
+      
+      begin
+        require 'caffeine'
+        Caffeine::MemCache.class_eval{ attr_reader :namespace  }
+      rescue          
+      end
+                 
       silence_warnings do
-        Object.const_set :CACHE, MemCache.new(config)
+        Object.const_set :CACHE, Object.const_defined?( :Caffeine ) ? Caffeine::MemCache.new( { :namespace => config[:namespace] } ) : MemCache.new(config) 
+                                 
       end

       CACHE.servers = Array(config.delete(:servers))
Index: vendor/plugins/cache_fu/lib/acts_as_cached/cache_methods.rb
===================================================================
--- vendor/plugins/cache_fu/lib/acts_as_cached/cache_methods.rb (revision 2337)
+++ vendor/plugins/cache_fu/lib/acts_as_cached/cache_methods.rb (working copy)
@@ -26,7 +26,7 @@
         id = args.first
       end

-      if (item = fetch_cache(id)).nil?
+      if (item = fetch_cache(id)).nil? || item == :MemCache_no_such_entry
         set_cache(id, block_given? ? yield : fetch_cachable_data(id), options[:ttl])
       else
         item
@@ -239,7 +239,7 @@
       set_cache
     end

-    # Lourens Naud?
+    # Lourens Naudé
     def expire_cache_with_associations(*associations_to_sweep)
       ((cache_options[:include] || []) + associations_to_sweep).flatten.uniq.compact.each do |assoc|
         Array(send(assoc)).compact.each { |item| item.expire_cache if item.respond_to?(:expire_cache) }

Warmup Mongrels on deploy

2007-05-19T02:08:00Z

... so your users don't have to

After each deployment with Capistrano, there's a slight delay on the first request to each of your Mongrel processes.

Any Libraries, GEMS, Models, Helpers, Controllers etc. not referenced by plugins or during config.inititalize ( and the callback variants ) in your environment is registered and loaded by the Rails Dependency mechanism on the first request.

This delay is relative to your Application size and the number of third party dependencies (2 seconds for me).

A Warmup task for your deployment recipe

CAUTION: The namespaces is Capistrano 2 specific, just drop them if you're using 1.x revisions.

This snippet would only execute IF

a configuration file ( mongrelcluster.yml ) exists in RAILSROOT/config
your production system has the Wget package installed ( standard on most Linux distros ).Modify this for your OS specific CLI HTTP client.

A single request is sent to each of your mongrel ports as defined in mongrel_cluster.yml. This may 404 depending on your application configuration ( subdomain based? ), in other words any status code is fine, 'Connection refused' not.

namespace :deploy do
  task :warmup, :roles => :app do
    require 'yaml'
     c = YAML.load_file(File.join('./', 'config', 'mongrel_cluster.yml')) rescue {}
     ((c["port"].to_i)...(c["port"].to_i + c["servers"].to_i)).each do |port|
       run 'wget localhost:%04d > /dev/null' % port  rescue 'No wget binary found'
       sleep(0.2)
     end unless c.empty?
  end
end

The call chain

I use the pattern below as my default App server deploy recipe.

namespace :deploy do
  task :default, :roles => :app do
    update
    compress_and_package_assets
    web:disable
    restart
    sleep(20)
    warmup
    sleep(2)
    web:enable
  end
end

Update code
Gzip and merge all JS and CSS assets
Disable App tier via Proxy
Fire up Mongrels and allow 20 seconds for them to do so
Warm them up
Sleep another 2 seconds for everything to settle down
Allow traffic via Proxy

Rails memory usage case study

2007-05-10T22:38:00Z

Stumbled across this interesting thread earlier today.

The remainder of this post assumes the following:

Production environment
Mongrel does not leak memory (Zed would have told you so if it did)
Latest stable Mongrel and FastThread
You're not using the usual culprits, RMagick and Ferret
You're not running any patched GC

The system

My staging system is CentOS 4 (64 bit) with 2GB RAM and runs 5 x PHP FCGI's, Nginx, Memcached, MySQL 5 and 8 x Mongrels. Staging Mongrel traffic only, medium traffic PHP forum.

Linux memory management

Here's a snapshot after just firing up mongrel_cluster:


[bookings@somehost ~]$  free -m
             total       used       free     shared    buffers     cached
Mem:          2003       1390        612          0        107        152
-/+ buffers/cache:       1131        871
Swap:         1992         81       1910

The -/+ buffers/cache is how much memory I have available, swap space is pretty much stock defaults for CentOS.

Here's a process snapshot of the Mongrels ( all idle, no traffic, no warm up, yet ):


[bookings@somehost ~]$ ps aux | grep mongrel
USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
mongrel   7659  2.7  4.8 139080 99840 ?      S    11:35   0:15 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8000 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8000.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7662  2.6  4.8 138648 99408 ?      S    11:35   0:15 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8001 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8001.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7665  2.7  4.6 134836 95560 ?      S    11:35   0:15 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8002 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8002.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7668  2.7  4.6 133684 94408 ?      S    11:35   0:15 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8003 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8003.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7671  2.7  4.7 135744 96540 ?      S    11:35   0:15 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8004 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8004.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7674  2.7  4.7 135768 96540 ?      S    11:35   0:15 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8005 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8005.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7677  2.7  4.6 133644 94404 ?      S    11:35   0:15 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8006 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8006.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7680  2.7  4.8 139284 100044 ?     S    11:36   0:15 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8007 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8007.pid -c /u/apps/bookings/current --user mongrel --group mongrel

Virtual Size

The Virtual Size (VIRT in 'top' output) for the process with PID 7659 is 139MB.

Total memory used by the process
INCLUDING any shared libraries (FreeImage, MySQL, Zlib etc.)
INCLUDING memory shared with other processes

A more verbose example of exactly which shared libs we require ( Rails AND Mongrel ):


[root@somehost bookings]# pmap -d 7659
7659:   /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8000 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8000.pid -c /u/apps/bookings/current --user mongrel --group mongrel
Address           Kbytes Mode  Offset           Device    Mapping
0000000000400000     780 r-x-- 0000000000000000 008:00003 ruby
00000000005c3000       8 rw--- 00000000000c3000 008:00003 ruby
00000000005c5000   43380 rwx-- 00000000005c5000 000:00000   [ anon ]
0000002a95556000      16 rw--- 0000002a95556000 000:00000   [ anon ]
0000002a95563000     404 rw--- 0000002a95563000 000:00000   [ anon ]
0000002a955c9000     704 rw--- 0000002a955c9000 000:00000   [ anon ]
0000002a95679000      12 r-x-- 0000000000000000 008:00003 digest.so
0000002a9567c000    1020 ----- 0000000000003000 008:00003 digest.so
0000002a9577b000       4 rw--- 0000000000002000 008:00003 digest.so
0000002a9577c000     232 r-x-- 0000000000000000 008:00003 openssl.so
0000002a957b6000    1020 ----- 000000000003a000 008:00003 openssl.so
0000002a958b5000      12 rw--- 0000000000039000 008:00003 openssl.so
0000002a958b8000       4 r-x-- 0000000000000000 008:00003 fcntl.so
0000002a958b9000    1020 ----- 0000000000001000 008:00003 fcntl.so
0000002a959b8000       4 rw--- 0000000000000000 008:00003 fcntl.so
0000002a959b9000    3548 rw--- 0000002a959b9000 000:00000   [ anon ]
0000002a95d30000      20 r-x-- 0000000000000000 008:00003 stringio.so
0000002a95d35000    1020 ----- 0000000000005000 008:00003 stringio.so
0000002a95e34000       4 rw--- 0000000000004000 008:00003 stringio.so
0000002a95e35000     112 r-x-- 0000000000000000 008:00003 syck.so
0000002a95e51000    1020 ----- 000000000001c000 008:00003 syck.so
0000002a95f50000       8 rw--- 000000000001b000 008:00003 syck.so
0000002a95f52000      40 r-x-- 0000000000000000 008:00003 socket.so
0000002a95f5c000    1024 ----- 000000000000a000 008:00003 socket.so
0000002a9605c000       4 rw--- 000000000000a000 008:00003 socket.so
0000002a9605d000      24 r-x-- 0000000000000000 008:00003 http11.so
0000002a96063000    1024 ----- 0000000000006000 008:00003 http11.so
0000002a96163000       4 rw--- 0000000000006000 008:00003 http11.so
0000002a96164000      16 r-x-- 0000000000000000 008:00003 fastthread.so
0000002a96168000    1020 ----- 0000000000004000 008:00003 fastthread.so
0000002a96267000       4 rw--- 0000000000003000 008:00003 fastthread.so
0000002a96268000      16 r-x-- 0000000000000000 008:00003 thread.so
0000002a9626c000    1020 ----- 0000000000004000 008:00003 thread.so
0000002a9636b000       4 rw--- 0000000000003000 008:00003 thread.so
0000002a9636c000      36 r-x-- 0000000000000000 008:00003 zlib.so
0000002a96375000    1020 ----- 0000000000009000 008:00003 zlib.so
0000002a96474000       4 rw--- 0000000000008000 008:00003 zlib.so
0000002a96475000       8 r-x-- 0000000000000000 008:00003 etc.so
0000002a96477000    1024 ----- 0000000000002000 008:00003 etc.so
0000002a96577000       4 rw--- 0000000000002000 008:00003 etc.so
0000002a96579000    4104 rw--- 0000002a96579000 000:00000   [ anon ]
0000002a96987000      40 r-x-- 0000000000000000 008:00003 libnss_files-2.3.4.so
0000002a96991000    1024 ----- 000000000000a000 008:00003 libnss_files-2.3.4.so
0000002a96a91000       8 rw--- 000000000000a000 008:00003 libnss_files-2.3.4.so
0000002a96a93000      16 r-x-- 0000000000000000 008:00003 strscan.so
0000002a96a97000    1024 ----- 0000000000004000 008:00003 strscan.so
0000002a96b97000       4 rw--- 0000000000004000 008:00003 strscan.so
0000002a96b98000     240 r-x-- 0000000000000000 008:00003 nkf.so
0000002a96bd4000    1024 ----- 000000000003c000 008:00003 nkf.so
0000002a96cd4000      16 rw--- 000000000003c000 008:00003 nkf.so
0000002a96cd8000       4 rw--- 0000002a96cd8000 000:00000   [ anon ]
0000002a96cd9000      40 r-x-- 0000000000000000 008:00003 bigdecimal.so
0000002a96ce3000    1024 ----- 000000000000a000 008:00003 bigdecimal.so
0000002a96de3000       4 rw--- 000000000000a000 008:00003 bigdecimal.so
0000002a96de5000    7384 rw--- 0000002a96de5000 000:00000   [ anon ]
0000002a9751b000       4 r-x-- 0000000000000000 008:00003 md5.so
0000002a9751c000    1020 ----- 0000000000001000 008:00003 md5.so
0000002a9761b000       4 rw--- 0000000000000000 008:00003 md5.so
0000002a9761c000      16 r-x-- 0000000000000000 008:00003 cparse.so
0000002a97620000    1020 ----- 0000000000004000 008:00003 cparse.so
0000002a9771f000       4 rw--- 0000000000003000 008:00003 cparse.so
0000002a97720000      16 r-x-- 0000000000000000 008:00003 iconv.so
0000002a97724000    1020 ----- 0000000000004000 008:00003 iconv.so
0000002a97823000       4 rw--- 0000000000003000 008:00003 iconv.so
0000002a97824000   13292 rw--- 0000002a97824000 000:00000   [ anon ]
0000002a9851f000       4 r-x-- 0000000000000000 008:00003 sha1.so
0000002a98520000    1020 ----- 0000000000001000 008:00003 sha1.so
0000002a9861f000       4 rw--- 0000000000000000 008:00003 sha1.so
0000002a98620000   23924 rw--- 0000002a98620000 000:00000   [ anon ]
0000002a99d7d000     916 r-x-- 0000000000000000 008:00003 mysql.so
0000002a99e62000    1020 ----- 00000000000e5000 008:00003 mysql.so
0000002a99f61000    1036 rw--- 00000000000e4000 008:00003 mysql.so
0000002a9a064000       8 rw--- 0000002a9a064000 000:00000   [ anon ]
000000320ac00000      84 r-x-- 0000000000000000 008:00003 ld-2.3.4.so
000000320ad14000       8 rw--- 0000000000014000 008:00003 ld-2.3.4.so
000000320ae00000    1196 r-x-- 0000000000000000 008:00003 libc-2.3.4.so
000000320af2b000    1024 ----- 000000000012b000 008:00003 libc-2.3.4.so
000000320b02b000       8 r---- 000000000012b000 008:00003 libc-2.3.4.so
000000320b02d000      12 rw--- 000000000012d000 008:00003 libc-2.3.4.so
000000320b030000      16 rw--- 000000320b030000 000:00000   [ anon ]
000000320b100000       8 r-x-- 0000000000000000 008:00003 libdl-2.3.4.so
000000320b102000    1020 ----- 0000000000002000 008:00003 libdl-2.3.4.so
000000320b201000       8 rw--- 0000000000001000 008:00003 libdl-2.3.4.so
000000320b300000     532 r-x-- 0000000000000000 008:00003 libm-2.3.4.so
000000320b385000    1020 ----- 0000000000085000 008:00003 libm-2.3.4.so
000000320b484000       8 rw--- 0000000000084000 008:00003 libm-2.3.4.so
000000320b500000      76 r-x-- 0000000000000000 008:00003 libz.so.1.2.1.2
000000320b513000    1020 ----- 0000000000013000 008:00003 libz.so.1.2.1.2
000000320b612000       4 rw--- 0000000000012000 008:00003 libz.so.1.2.1.2
000000320b700000      68 r-x-- 0000000000000000 008:00003 libresolv-2.3.4.so
000000320b711000    1024 ----- 0000000000011000 008:00003 libresolv-2.3.4.so
000000320b811000       8 rw--- 0000000000011000 008:00003 libresolv-2.3.4.so
000000320b813000       8 rw--- 000000320b813000 000:00000   [ anon ]
000000320b900000      20 r-x-- 0000000000000000 008:00003 libcrypt-2.3.4.so
000000320b905000    1020 ----- 0000000000005000 008:00003 libcrypt-2.3.4.so
000000320ba04000       8 rw--- 0000000000004000 008:00003 libcrypt-2.3.4.so
000000320ba06000     184 rw--- 000000320ba06000 000:00000   [ anon ]
000000320bd00000      80 r-x-- 0000000000000000 008:00003 libnsl-2.3.4.so
000000320bd14000    1020 ----- 0000000000014000 008:00003 libnsl-2.3.4.so
000000320be13000       8 rw--- 0000000000013000 008:00003 libnsl-2.3.4.so
000000320be15000       8 rw--- 000000320be15000 000:00000   [ anon ]
000000320bf00000       8 r-x-- 0000000000000000 008:00003 libcom_err.so.2.1
000000320bf02000    1020 ----- 0000000000002000 008:00003 libcom_err.so.2.1
000000320c001000       4 rw--- 0000000000001000 008:00003 libcom_err.so.2.1
000000320c300000     436 r-x-- 0000000000000000 008:00003 libkrb5.so.3.2
000000320c36d000    1024 ----- 000000000006d000 008:00003 libkrb5.so.3.2
000000320c46d000      16 rw--- 000000000006d000 008:00003 libkrb5.so.3.2
000000320c500000    1080 r-x-- 0000000000000000 008:00003 libcrypto.so.0.9.7a
000000320c60e000    1024 ----- 000000000010e000 008:00003 libcrypto.so.0.9.7a
000000320c70e000     120 rw--- 000000000010e000 008:00003 libcrypto.so.0.9.7a
000000320c72c000      16 rw--- 000000320c72c000 000:00000   [ anon ]
000000320c800000     136 r-x-- 0000000000000000 008:00003 libk5crypto.so.3.0
000000320c822000    1020 ----- 0000000000022000 008:00003 libk5crypto.so.3.0
000000320c921000       8 rw--- 0000000000021000 008:00003 libk5crypto.so.3.0
000000320ca00000      84 r-x-- 0000000000000000 008:00003 libgssapi_krb5.so.2.2
000000320ca15000    1024 ----- 0000000000015000 008:00003 libgssapi_krb5.so.2.2
000000320cb15000       4 rw--- 0000000000015000 008:00003 libgssapi_krb5.so.2.2
000000320ce00000     216 r-x-- 0000000000000000 008:00003 libssl.so.0.9.7a
000000320ce36000    1024 ----- 0000000000036000 008:00003 libssl.so.0.9.7a
000000320cf36000      20 rw--- 0000000000036000 008:00003 libssl.so.0.9.7a
000000320cf3b000       4 rw--- 000000320cf3b000 000:00000   [ anon ]
0000007fbffa7000     356 rw--- 0000007fbffa7000 000:00000   [ stack ]
ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ]
mapped: 147272K    writeable/private: 98744K    shared: 0K

Resident Size

The Resident Size (RES in 'top' output) for the process with PID 7659 is 99MB. This is the net amount amount of memory consumed by the process.

The app

Whoa, that's a lot of memory! rake:stats ...


(in /Users/lourens/projects/bookings)
+----------------------+-------+-------+---------+---------+-----+-------+
| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Controllers          |  5622 |  4351 |      99 |     604 |   6 |     5 |
| Helpers              |  2493 |  1909 |       0 |     467 |   0 |     2 |
| Models               |  5654 |  4067 |     209 |     685 |   3 |     3 |
| Libraries            |  2258 |  1476 |      42 |     210 |   5 |     5 |
| APIs                 |     0 |     0 |       0 |       0 |   0 |     0 |
| Components           |     0 |     0 |       0 |       0 |   0 |     0 |
| Model specs          | 14977 | 11275 |       0 |     131 |   0 |    84 |
| View specs           |     0 |     0 |       0 |       0 |   0 |     0 |
| Controller specs     | 24538 | 18954 |       0 |       0 |   0 |     0 |
| Helper specs         |     0 |     0 |       0 |       0 |   0 |     0 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total                | 55542 | 42032 |     350 |    2097 |   5 |    18 |
+----------------------+-------+-------+---------+---------+-----+-------+
  Code LOC: 11803     Test LOC: 30229     Code to Test Ratio: 1:2.6

Memcached is used extensively.

My memory hogs

In addition to application code and stock Rails:

PDF Writer and TZinfo gems
Globalize ( 8 languages, 1400 translations per language, preloaded in config.after_initialize ) and another 20 odd plugins
Per request local cache for cache_fu

Memory usage for remainder of the stack

Nginx, neglible:


[root@somehost bookings]# ps aux | grep nginx
root     17280  0.0  0.0 18192  548 ?        Ss   Apr22   0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
nginx    17281  0.0  0.1 18948 2400 ?        S    Apr22  13:12 nginx: worker process                                          
nginx    17282  0.0  0.1 18828 2196 ?        S    Apr22  12:58 nginx: worker process

Mysql, pretty standard:


[root@somehost bookings]# ps aux | grep mysql
root     10187  0.0  0.0 53840 1316 ?        S    May03   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/mysql/data --pid-file=/mysql/data/somehost.com.pid
mysql    10226  0.1  3.5 733792 73524 ?      Sl   May03  18:24 /usr/sbin/mysqld --basedir=/ --datadir=/mysql/data --user=mysql --pid-file=/mysql/data/somehost.com.pid --skip-external-locking --port=3306 --socket=/var/lib/mysql/mysql.sock

PHP:


[root@somehost bookings]# ps aux | grep php
505       8352  0.0  0.0 58880 1172 ?        Ss   Mar28   0:00 /usr/local/bin/php
505       7597  0.8  0.6 71172 14304 ?       S    11:29   0:23 /usr/local/bin/php
505       7750  0.9  0.6 71160 14276 ?       S    11:54   0:13 /usr/local/bin/php
505       7754  0.9  0.6 71232 14272 ?       S    11:55   0:11 /usr/local/bin/php
505       7820  1.1  0.4 65284 8436 ?        S    12:06   0:07 /usr/local/bin/php
505       7823  1.0  0.6 71232 14288 ?       S    12:08   0:04 /usr/local/bin/php

Memcached:


[root@somehost bookings]# ps aux | grep memcached
502       5354  0.0  1.3 45784 28320 ?       Ss   May04   0:13 memcached -d -m 128 -p 11211 -u memcached

After a decent warmup

Snapshot of the Mongrels - 4000 requests (concurrency 8, 250 calls, rate of 2) across the Nginx/MySQL/Mongrel stack :


[root@somehost bookings]# ps aux | grep mongrel
mongrel   7659  0.7  5.8 164704 120156 ?     S    11:35   1:20 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8000 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8000.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7662  0.7  5.8 164668 120128 ?     S    11:35   1:20 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8001 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8001.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7665  0.7  5.8 164728 120180 ?     S    11:35   1:20 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8002 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8002.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7668  0.8  5.8 165300 120668 ?     S    11:35   1:21 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8003 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8003.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7671  0.8  5.8 164636 120084 ?     S    11:35   1:20 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8004 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8004.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7674  0.8  5.8 164988 120400 ?     S    11:35   1:20 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8005 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8005.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7677  0.7  5.8 164772 120224 ?     S    11:35   1:20 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8006 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8006.pid -c /u/apps/bookings/current --user mongrel --group mongrel
mongrel   7680  0.8  5.8 164840 120188 ?     S    11:36   1:20 /usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -p 8007 -a 127.0.0.1 -P /u/apps/bookings/current/log/mongrel.8007.pid -c /u/apps/bookings/current --user mongrel --group mongrel

Resident Size increased by 20MB to 24MB per process.

Why memory increases after warm up

Any gems or libs referenced by your plugins, environment files and config.after_initialize blocks will be required on process startup.
The Dependencies mechanism in Rails >= 1.2 kicks in after the first request, during which most of your application code and gems/libs not required during initialization is loaded.
String count in ObjectSpace will increase dramaticly: ERB templates/partials, routes, lib/file paths, generated SQL and Marshalled data if you're using a cache backend.

Your process size after proper warmup should remain more or less constant (give or take for GC).

What to do if you suspect memory leaks

Bleak House

An excellent tool for polling your production environment at set intervals.A pure C (custom binary) and ObjectSpace version is available.

Mongrel "Dash-Bee" logging

Start Mongrel with the '-B' switch. This yields a few log files in RAILSROOT/log/mongreldebug, of which you should inspect the following :

objects.log - A log of all the Objects currently in ObjectSpace.Diff this per request count and watch for a growing Delta.
files.log - Lists open file descriptors.Useful if your application is IO bound.

Finding open file descriptors

'lsof' is a neat utility for listing open ports, sockets and files on a per process basis.


[root@somehost bookings]# /usr/sbin/lsof -p 7659
COMMAND    PID    USER   FD   TYPE             DEVICE     SIZE     NODE NAME
mongrel_r 7659 mongrel  cwd    DIR                8,3     4096  1639306 /u/apps/bookings/releases/20070510100305
mongrel_r 7659 mongrel  rtd    DIR                8,3     4096        2 /
mongrel_r 7659 mongrel  txt    REG                8,3  2493404  1696180 /usr/local/bin/ruby
mongrel_r 7659 mongrel  mem    REG                8,3    32267  1926292 /usr/local/lib/ruby/1.8/x86_64-linux/digest.so
mongrel_r 7659 mongrel  mem    REG                8,3   832033  1926293 /usr/local/lib/ruby/1.8/x86_64-linux/openssl.so
mongrel_r 7659 mongrel  mem    REG                8,3    10544  1926290 /usr/local/lib/ruby/1.8/x86_64-linux/fcntl.so
mongrel_r 7659 mongrel  mem    REG                8,3    51414  1926305 /usr/local/lib/ruby/1.8/x86_64-linux/stringio.so
mongrel_r 7659 mongrel  mem    REG                8,3   341425  1926303 /usr/local/lib/ruby/1.8/x86_64-linux/syck.so
mongrel_r 7659 mongrel  mem    REG                8,3   119273  1926301 /usr/local/lib/ruby/1.8/x86_64-linux/socket.so
mongrel_r 7659 mongrel  mem    REG                8,3    74815  2330154 /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/http11.so
mongrel_r 7659 mongrel  mem    REG                8,3    46134  1920694 /usr/local/lib/ruby/gems/1.8/gems/fastthread-1.0/lib/fastthread.so
mongrel_r 7659 mongrel  mem    REG                8,3    45517  1926304 /usr/local/lib/ruby/1.8/x86_64-linux/thread.so
mongrel_r 7659 mongrel  mem    REG                8,3   108943  1926307 /usr/local/lib/ruby/1.8/x86_64-linux/zlib.so
mongrel_r 7659 mongrel  mem    REG                8,3    24565  1926302 /usr/local/lib/ruby/1.8/x86_64-linux/etc.so
mongrel_r 7659 mongrel  mem    REG                8,3    56902  2588702 /lib64/libnss_files-2.3.4.so
mongrel_r 7659 mongrel  mem    REG                8,3    47492  1926298 /usr/local/lib/ruby/1.8/x86_64-linux/strscan.so
mongrel_r 7659 mongrel  mem    REG                8,3   424522  1926299 /usr/local/lib/ruby/1.8/x86_64-linux/nkf.so
mongrel_r 7659 mongrel  mem    REG                8,3   131340  1926308 /usr/local/lib/ruby/1.8/x86_64-linux/bigdecimal.so
mongrel_r 7659 mongrel  mem    REG                8,3    12623  1926296 /usr/local/lib/ruby/1.8/x86_64-linux/digest/md5.so
mongrel_r 7659 mongrel  mem    REG                8,3    37234  1926306 /usr/local/lib/ruby/1.8/x86_64-linux/racc/cparse.so
mongrel_r 7659 mongrel  mem    REG                8,3    40134  1926291 /usr/local/lib/ruby/1.8/x86_64-linux/iconv.so
mongrel_r 7659 mongrel  mem    REG                8,3    13615  1926295 /usr/local/lib/ruby/1.8/x86_64-linux/digest/sha1.so
mongrel_r 7659 mongrel  mem    REG                8,3  4756583  1920155 /usr/local/lib/ruby/site_ruby/1.8/x86_64-linux/mysql.so
mongrel_r 7659 mongrel  mem    REG                8,3    21546  1721087 /usr/lib64/gconv/gconv-modules.cache
mongrel_r 7659 mongrel  mem    REG                8,3    24492   359548 /home/mongrel/.ruby_inline/Inline_ImageScience_aa58.so
mongrel_r 7659 mongrel  mem    REG                8,3  1547144  1697450 /usr/lib/libfreeimage-3.9.3.so
mongrel_r 7659 mongrel  mem    REG                8,3    47496  2588909 /lib64/libgcc_s-3.4.6-20060404.so.1
mongrel_r 7659 mongrel  mem    REG                8,3   105080  2588770 /lib64/ld-2.3.4.so
mongrel_r 7659 mongrel  mem    REG                8,3  1493409  2588771 /lib64/tls/libc-2.3.4.so
mongrel_r 7659 mongrel  mem    REG                8,3    17943  2588904 /lib64/libdl-2.3.4.so
mongrel_r 7659 mongrel  mem    REG                8,3   613297  2588908 /lib64/tls/libm-2.3.4.so
mongrel_r 7659 mongrel  mem    REG                8,3    79336  1701661 /usr/lib64/libz.so.1.2.1.2
mongrel_r 7659 mongrel  mem    REG                8,3    91412  2588903 /lib64/libresolv-2.3.4.so
mongrel_r 7659 mongrel  mem    REG                8,3    30070  2588910 /lib64/libcrypt-2.3.4.so
mongrel_r 7659 mongrel  mem    REG                8,3   107187  2588726 /lib64/libnsl-2.3.4.so
mongrel_r 7659 mongrel  mem    REG                8,3    10384  2588902 /lib64/libcom_err.so.2.1
mongrel_r 7659 mongrel  mem    REG                8,3   464040  1701659 /usr/lib64/libkrb5.so.3.2
mongrel_r 7659 mongrel  mem    REG                8,3  1230232  2588905 /lib64/libcrypto.so.0.9.7a
mongrel_r 7659 mongrel  mem    REG                8,3   145456  1698080 /usr/lib64/libk5crypto.so.3.0
mongrel_r 7659 mongrel  mem    REG                8,3    93832  1701660 /usr/lib64/libgssapi_krb5.so.2.2
mongrel_r 7659 mongrel  mem    REG                8,3   910744  1697214 /usr/lib64/libstdc++.so.6.0.3
mongrel_r 7659 mongrel  mem    REG                8,3   244320  2588906 /lib64/libssl.so.0.9.7a
mongrel_r 7659 mongrel    0r   CHR                1,3              1982 /dev/null
mongrel_r 7659 mongrel    1w   REG                8,3   572768  1579794 /u/apps/bookings/shared/log/mongrel.log
mongrel_r 7659 mongrel    2w   REG                8,3   572768  1579794 /u/apps/bookings/shared/log/mongrel.log
mongrel_r 7659 mongrel    3u  IPv4           14667600               TCP localhost.localdomain:8000 (LISTEN)
mongrel_r 7659 mongrel    4w   REG                8,3 68750821  1579790 /u/apps/bookings/shared/log/production.log
mongrel_r 7659 mongrel    5u  IPv4           14667733               TCP localhost.localdomain:39863->localhost.localdomain:11211 (ESTABLISHED)
mongrel_r 7659 mongrel    6u  unix 0x0000010072f89380          14690356 socket
mongrel_r 7659 mongrel    7w   REG                8,3       57  1556510 /u/apps/bookings/shared/log/memory.log
mongrel_r 7659 mongrel    9r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   10r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   11r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   12r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   13r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   14r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   15r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   16r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   17r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   18r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   19r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   20r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   21r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   22r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   23r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   24r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   25r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   26r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   27r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   28r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   29r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   30r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   31r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   32r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   33r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   34r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   35r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   36r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   37r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   38r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   39r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   40r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   41r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   42r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   43r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   44r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   45r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   46r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   47r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   48r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   49r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   50r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   51r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   52r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   53r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   54r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   55r   DIR                8,3     4096  1823783 /u/apps/bookings/releases/20070510100305/public/stylesheets
mongrel_r 7659 mongrel   56r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   57r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   58r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   59r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   60r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   61r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   62r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   63r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   64r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   65r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   66r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   67r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   68r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   69r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts
mongrel_r 7659 mongrel   70r   DIR                8,3     4096  2528355 /u/apps/bookings/releases/20070510100305/public/javascripts

MemoryProfiler

Scott Laird and Ryan Davis hacked up a memory leak spotter with a very useful strings logger. Rails plugin available here.

WeakRef

The WeakRef Standard Library class can be used to clean up in tough spots where the GC doesn't clean up properly.In process caches etc. See WeakHash for in depth usage.

Custom logging

I use the following pattern ( credits to BleakHouse ) for logging without polluting the application with ActionController filters:

class Dispatcher
  class << self
    def reset_after_dispatch_with_globalize
      translator = Globalize::DbViewTranslator.instance     
      ActiveRecord::Base.logger.info "Cache size: #{translator.cache_size}, Cache total hits: #{translator.cache_total_hits}, Cache total queries: #{translator.cache_total_queries}, Cache keys total: #{translator.instance_variable_get(:@cache).keys.size}"
      reset_after_dispatch() 
    end
    alias_method_chain :reset_after_dispatch, :globalize

  end
 end

Conclusion

Rails is greedy on Memory, however :

If you spent the time developing and testing an application, why cut back on resources if a Dual Core box with 2GB RAM can be rented for $130 to $200 p/m at many respectable hosts?
Many opt for VPS solutions, which pretty much require at least some sysadmin skills ... is the time and monitoring spent on avoiding swap space for anything more than a blog really worth it?
I like the Container/Slice approach offered by Joyent and Engine Yard a lot from a administration and management perspective, but can't help feeling restricted by the memory offered ( Memcached? ).Adding an additional 1GB to a dedicated box will cost you $15 to $30 at most providers ... but in Container/Slice terms that's an additional container, worst case scenario.
I assume many large Rails applications have a Resident Size of 100MB to 130MB per process average, anyone care to chime in and share some production memory usage for others to have a baseline before loosing sleep over memory consumption?

Volatile memoization for Marshalled AR objects

2007-05-08T22:26:00Z

The Ruby memcache-client gem uses the Marshal library to serialize ruby objects for storage in Memcached.

By default, all instance variables is marshalled as well, which pretty much defeats instance variable based memoization eg.

  def to_param; @to_param ||= "#{self.id}-#{self.name.to_url}" end

Usage scenario

An account based application where almost all Models belongs to an account:

  class Account < ActiveRecord::Base
     acts_as_cached :version => 1, :include => [:subscription, :setting, :logo, :domain, :active_theme], :ttl => 1.day

     belongs_to :domain
     has_many :bookings, :class_name => 'PropertyBooking', :include => [:date, :customer], :foreign_key => :account_id, :dependent => :delete_all, :order => 'bookings.created_at DESC', :extend => BookingsExtension
    #define other relationships
  end

We set the account instance in a before filter for every request:

  class AccountController < ApplicationController
     attr_accessor :account 
     before_filter :account_from_subdomain

     protected

      def account_from_subdomain   
         @account = Account.get_cache([account_subdomain(),request.domain].join('.'))
         if @account.nil?
           account_not_found  
         else
          @account.disabled? ? account_disabled() : account_found()
        end    
     end         

  end

From time to time we need to reference the account belongs_to association to load/query other data scoped to the particular account:

  class Booking < ActiveRecord::Base
    #associations etc.

    #verify subscription limit before saving another booking
    def before_create; self.account.subscription.bookings_this_cycle? end 

  end

Headaches

Simply caching an account alongside a child Model forces us to implement excessive sweepers to preserve data integrity and also duplicates data, which isn't desireable:

class Booking < ActiveRecord::Base
  #cache account alongside this model  
   acts_as_cached :version => 1, :include => [:account, :date, :progress, :payment], :ttl => 2.hours

end

We can override the reader/accessor method, but this results in excessive Memcache lookups if the related model is referenced multiple times per request:

 class Booking < ActiveRecord::Base
  #do not cache the account alongside this model
  acts_as_cached :version => 1, :include => [:date, :progress, :payment], :ttl => 2.hours

  #potential multiple lookups
  def account; Account.get_cache(account_id) unless account_id.nil? end
end

The solution

We can use the after_initialize callback to clear an instance variable used for memoization:

 class Booking < ActiveRecord::Base
  #do not cache the account alongside this model
  acts_as_cached :version => 1, :include => [:date, :progress, :payment], :ttl => 2.hours

  #memoization, only called once per lifecycle/request
  def account; @cached_account ||= Account.get_cache(account_id) unless account_id.nil? end

  #AR callback
  def after_initialize; @cached_account = nil end

end

In theory either afterinitialize OR afterfind can be used to accomplish this.

You need to ensure your cache backend reference these callbacks to mimick and replicate the standard ActiveRecord behaviour.

I use the following snippet for cache_fu:

module ActsAsCached
   module ClassMethods

       def fetch_cache(id)
         return if ActsAsCached.config[:skip_gets]

           autoload_missing_constants do 
              data = cache_store(:get, cache_key(id))
              [:after_initialize, :after_find].each{|c| data.send(c) if data.respond_to? c }
              data
          end
      end
   end
end

memcache-client can be monkey patched if you rolled your own store using the CACHE.get API method:

class MemCache
  def get(key, raw = false)
    server, cache_key = request_setup key

    value = if @multithread then
              threadsafe_cache_get server, cache_key
            else
              cache_get server, cache_key
            end

    return nil if value.nil?

    unless raw
       value = Marshal.load value     
       [:after_initialize, :after_find].each{|c| value.send(c) if value.respond_to? c } 
    end

    return value
  rescue TypeError, SocketError, SystemCallError, IOError => err
    handle_error server, err
  end
end

Alternatives

The Marshal library define callbacks for more complex marshalling:

  class SomeModel < ActiveRecord::Base

     #prepare for serialization
     def marshal_dump
     end

     #do something when deserialized
     def marshal_load( serialized_string )
     end

  end

Partially bypass ActiveRecord instantiation when using Memcached

2007-04-24T00:57:00Z

With the recent scaling woes and Blaine Cook's Scaling Twitter talk, I think slide 23 is a welcome, and bold, statement.

Memcahed is generally used as an intermediate cache to lessen database load, and especially more crucial when using ORM ( eg. ActiveRecord ) as avoiding/bypassing object instantiation will give you additional mileage in the App tier as well.

The context:

Three example models: Account, Customer and Booking ( some omitted for simplicity ).

class Account < ActiveRecord::Base
  acts_as_cached :version => 1, :include => [:omitted], :ttl => 2.hours

  has_many :customers
  has_many :bookings
  has_many :contacts
  has_many :addresses
end

class Customer < ActiveRecord::Base
  acts_as_cached :version => 1, :include => [:address, :contact, :bookings, :account], :ttl => 2.hours 

  belongs_to :account
  has_one :address, :as => :addressable, :dependent => :destroy  
  has_one :contact, :as => :contactable, :dependent => :destroy
end

class Booking < ActiveRecord::Base
  acts_as_cached :version => 1, :include => [:omitted], :ttl => 2.hours[:date, :progress, :user, :customer, :events, :booking_extras, :booking_products, :notes, :payment, :account, :coupon], :ttl => 2.hours

  belongs_to :customer
end

The problem

We are caching the Customer model, with its direct and most often used associations (Contact, Address and Bookings) and would like to maintain this integrity, without instantiating additional AR objects and related associations when referencing all Customers or Bookings for a given Account:

  account.customers.to_a #or account.customers(true)
  account.bookings.to_a #or account.bookings(true)

However, Rails do have an association proxy reader method:

   account.customer_ids # => [1]
  account.booking_ids # => [6, 1, 5, 2, 3]

... which isn't very helpful if we'd like a specifc subset of customer or booking identifiers.

The solution

Extend ActiveRecord::Base with a find_ids singleton method, with the exact same usage as AR::Base#find, but never instantiates any objects.We only fetch the ID's from the raw connection:

module ActiveRecord 
  class Base  

   class << self
    def find_ids(*args)
      options = extract_options_from_args!(args)
      logger.debug("Find by ID:" + options.inspect)
      validate_find_options(options)      

      case args.first
        when :first then find_initial_id(options)
        when :all   then find_every_id(options)
      end
    end  

    def find_by_sql_ids(sql)
      connection.select_all(sanitize_sql(sql), "#{name} Load").collect! { |record| record['id'] }
    end    

    private

    def find_initial_id(options)
      options.update(:limit => 1) unless options[:include]
      find_every_id(options).first
    end

    def find_every_id(options)
      records = scoped?(:find, :include) || options[:include] ?
        find_with_associations_ids(options) : 
        find_by_sql_ids(construct_finder_sql(options))
      records
    end
    end        
  end  
end

module ActiveRecord
  module Associations
    module ClassMethods
      def find_with_associations_ids(options = {})
        catch :invalid_query do
          join_dependency = JoinDependency.new(self, merge_includes(scope(:find, :include), options[:include]), options[:joins])
          logger.debug("All rows: " + select_all_rows(options, join_dependency).inspect)
          return select_all_rows(options, join_dependency).collect { |row| row[join_dependency.joins.first.aliased_primary_key] }
        end
        []
      end
    end  
  end
end

Usage examples

Following is an association extension that illustrates compatibility with AR::Base#find :

module BookingsExtension
  def upcoming( page = 1) self.find_ids(:all, default_find_options(page)) end

  def recent( page = 1 ) self.find_ids(:all, default_find_options(page).merge!({:order => 'booking_dates.date_from DESC'})) end

  def by_status(status = 'pending', page = 1)
    self.find_ids(:all, default_find_options(page).merge!(:conditions => ['bookings.status = ?', status]))     
  end

    def since(date = Time.now.utc, page = 1)
    self.find_ids(:all, default_find_options(page).merge!(:conditions => ["bookings.status != ? AND booking_dates.date_from >= ?", 'in_progress', date.to_s(:db)]))      
  end  

  def until(date = Time.now.utc, page = 1)
    self.find_ids(:all, default_find_options(page).merge!(:conditions => ["bookings.status != ? AND booking_dates.date_from <= ?", 'in_progress', date.to_s(:db)]))        
  end

  def by_user(user, page = 1)
    self.find_ids(:all, default_find_options(page).merge!(:conditions => ["bookings.status != ? AND bookings.user_id = ?",'in_progress',user.id]))
  end

  def by_reference( reference, page = 1 )
    self.find_ids(:all, default_find_options(page).merge!(:conditions => ["bookings.status != ? AND bookings.reference = ?",'in_progress',reference]))
  end  

  def default_find_options(page)
    { :include => [:customer, :date], :conditions => ['bookings.status != ?', 'in_progress'], :order => 'bookings.created_at DESC', :page => { :size => 10, :current => page, :first => 1 } }
  end
end</code></pre>

<h2>Standalone examples:</h2>

<pre><code class="ruby">  account.bookings.by_status(:pending).to_a  # => ["2", "1"]

  account.bookings.by_user( User.get_cache(1) ).to_a # => ["2", "1", "3"]</code></pre>

<h2>Memcached friendly examples</h2>

<p>I use <a href="http://require.errtheblog.com/plugins/browser/cache_fu">cache<em>fu</a> to interface with memcache-client.The <a href="http://require.errtheblog.com/plugins/browser/cache_fu/defaults/extensions.rb.default">multi</em>get_cache</a> extension method is particularly useful here:</p>

<pre><code class="ruby">  Booking.multi_get_cache( ["2", "1", "3"] ) # => lots of output

In the above example we are attempting to fetch Bookings with ID 1..3 from Memcached.Should the objects already be cached, we only had the DB overhead of 1 relatively cheap query while maintaining cache integrity without duplicating any processing or data.

Conclusion

The above is a slight anti-pattern, which true the 90/10 principle, would only ever be useful to those users of the framework with Memcached in their production stack.

It's OK to break free from constraints, religious DRY development that may shoot you in the foot later and even denormalize, as per Blaine's slide number 23, if and when the framework doesn't natively solve your problem ( performance? design constraints? production environment? ) at hand.

DRY association sweeping with acts_as_cached

2007-01-11T02:39:00Z

Cherry picking exactly which models and associations to cache with Acts as Cached is dead simple.However, cached object associations quickly result in repetitive sweeping snippets.

How to cache associations

class Address < ActiveRecord::Base

  acts_as_cached :version => 1, :include => [:country] 
  belongs_to :addressable, :polymorphic => true #we can't do find :include => X with polymorhic associations
  belongs_to :country, :class_name => 'Globalize::Country', :foreign_key => 'country_id'

end

The above snippet will execute the following behind the scenes:

Address.get_cache(1) #Address.find(1, :include => [ :country ])

How to sweep associations

Overriding accessors

  class Address < ActiveRecord::Base

    def country
       Globalize::Country.get_cache( country_id ) unless country_id.nil?
    end

  end

Just works, always fresh but has the overhead of a ridiculous amount of CACHE.get lookups

Manually sweeping

  class Address < ActiveRecord::Base

    after_save :cache_sweeper
    after_destroy :cache_sweeper

    def cache_sweeper
       expire_cache
       country.expire_cache
    end

  end

and

  class Globalize::Country < ActiveRecord::Base

    has_many :addresses, :as => :addressable

    after_save :cache_sweeper
    after_destroy :cache_sweeper

    def cache_sweeper
       expire_cache
       addresses.each{|a| a.expire_cache }
    end

  end

Very bloated for complex relationships.

But there's a better way

Add the following to actsas_cached's InstanceMethods or Monkey Patch/moduleeval in your environment.

         def expire_cache_with_associations( *associations_to_sweep )
          ((self.class.cache_options[:include] || []) + associations_to_sweep).flatten.uniq.compact.each do |assoc|
            macro = self.class.reflect_on_association(assoc.to_sym).macro
            macro == :has_many ? self.send(assoc.to_sym).each{|r| r.expire_cache unless r.nil? } : self.send(assoc.to_sym).expire_cache unless self.send(assoc.to_sym).nil?
          end 
          expire_cache
        end

This snippet will sweep associations passed in the :includes config hash option, but also allows for additional associations to be merged in later on.

A DRY and refactored example

  class Address < ActiveRecord::Base
    acts_as_cached :version => 2, :include => [:country]

    #sweeps country    
    after_save :expire_cache_with_associations
    after_destroy :expire_cache_with_associations

  end

and

  class Globalize::Country < ActiveRecord::Base
    acts_as_cached :version => 1
    has_many :addresses, :as => :addressable

    after_save :cache_sweeper
    after_destroy :cache_sweeper

    #We don't cache all addresses with the country as one country can grow to a potentially
    #large number of adresses ... we just sweep to update the cached country for each address
    def cache_sweeper;  expire_cache_with_associations(:addresses) end

  end

Sweeping polymorphic associations

class Address < ActiveRecord::Base
  acts_as_cached :version => 1, :include => [:country] 
  belongs_to :addressable, :polymorphic => true 
  belongs_to :country, :class_name => 'Globalize::Country', :foreign_key => 'country_id'

  after_save     :cache_sweeper
  before_destroy :cache_sweeper

  protected  

  #sweep the related addressable entitiy
  def cache_sweeper; expire_cache_with_associations(:addressable) end

end

class Customer < ActiveRecord::Base
  acts_as_cached :version => 1, :include => [:address, :contact] 

  has_one :address, :as => :addressable, :dependent => :destroy  
  has_one :contact, :as => :contactable, :dependent => :destroy

  #nothing fancy here
  after_save     :expire_cache_with_associations
  before_destroy :expire_cache_with_associations
end  

class Contact < ActiveRecord::Base
  acts_as_cached :version => 1 
  belongs_to :contactable, :polymorphic => true

  after_save     :cache_sweeper
  before_destroy :cache_sweeper

  protected

  #sweep related contactable entitiy
  def cache_sweeper; expire_cache_with_associations(:contactable) end

end

Unobtrusive around filter profile snippet

2006-12-27T05:00:00Z

Drop into environment.rb or other required library.

require 'profiler'

class ProfilingFilter

  def before( controller )
    Profiler__::start_profile 
  end

  def after( controller )
    Profiler__::stop_profile
    Profiler__::print_profile(File::new("#{RAILS_ROOT}/log/profiler.log", 'w'))  
  end

end

Usage

Something hogs?

class SomeController < ApplicationController
  around_filter ProfilingFilter.new, :only => :slow_action

  def slow_action; end

  def speedy; end

  def zippy; end

end

Output dumped to log/profiler.log

Preloading Globalize view translations in production

2006-12-27T04:43:00Z

When you've properly extracted and translated view translations from your source code, here's a snippet to preload them all before the first request served in Production.

Dispatcher.to_prepare :globalize_view_translations do
  if RAILS_ENV == 'production' 
    translator = Globalize::DbViewTranslator.instance
      #Or whichever other means used to keep track of app specific supported languages 
      SupportedLanguage.find(:all).each do |lang|
        ViewTranslation.find(:all, :conditions => ['globalize_translations.language_id = ?', lang.language_id]).each do |tr|
          translator.send(:cache_add, tr.tr_key, tr.language, tr.pluralization_index, tr.text)
        end  
    end  
  end  
end

Dispatcher.to_prepare

An addition to 1.2 and Edge for quite some time.This callback is referenced

before the first request in Production
before each request in Development

Most notably initializes Observer instances

Globalize::DbTranslate with :include and no base language patch

2006-12-22T18:07:00Z

Globalize DB translations currently doesn't support a :select find option, which breaks hasmany :trough => atranslateable_model associations.

I've refactored the Globalize::DbTranslate module to an AR::Associations pattern, override Model.find to always preload translations whilst removing all of the custom finder SQL.

Benefits

Supports all of the ActiveRecord::Base#find functionality
Plugs in with existing globalize table structures (no migrations required)
There is no Base Locale.Edit or create in any language, any other language always retrieves a valid facet, albeit not always in the active Locale.
Retrieves all facet translations for a given model at once, thus Locale switching doesn't incur a database roundtrip overhead.

Caveats

I removed a tiny bit of functionality to allow me to focus on a initial prototype.

Bidi strings temp. removed
Removed piggy backing find option
Works best with only one Locale per request, which is fine in most situations.
The no Base Locale setup might not be ideal for all use cases.
Only tested with Edge (production, test and development envs)
Globalize unit test helper chokes with the Dependencies mechanism and I couldn't yet get the unit tests to run.

The rundown

AR association and Translateable.find

Setup an association in the translates() hook:

          has_many :translations, :class_name => 'Globalize::ModelTranslation', :conditions => ['globalize_translations.table_name = ?', table_name], :foreign_key => 'item_id'

Override Translateable.find, scoped to always prefetch translations:

   alias_method :old_find, :find
   def find(*args)
      options = args.last.is_a?(Hash) ? args.last : {}
      if (options.has_key?(:untranslated) && options[:untranslated] == true)
          old_find(*args)
      else
         with_scope( :find => { :include => :translations }) do
            old_find(*args)
         end
       end  
    end

Cherry pick facet content

     def find_translation_for_facet( facet )
         translations ||= []
         #highest priority, find for current language
         translations << self.translations.detect{|tr| tr.facet == facet.to_s && tr.language_id == Locale.language.id }
         #do we have a non-blank attribute in the model table?
         translations << read_attribute(facet)
         #find the first translation, for any language
         translations << self.translations.detect{|tr| tr.facet == facet.to_s && !tr.text.nil? }

         #we don't want nil or empty string
         translations.reject!(&:blank?)
         translations.empty? ? nil : (translations.first.respond_to?(:text) ? translations.first.text : translations.first)                          
      end

The patch

Grab this patch and give it a whirl.

Feedback on this? Performance implications? Backwards compatibility?

Using simply_helpful with rspec controller tests

2006-11-23T23:16:00Z

Rspec currently doesn't play well on Edge Rails with the simply_helpful plugin.

The cause

simply_helpful plugin hook (init.rb):

require 'simply_helpful'
ActionController::Base.send :include, SimplyHelpful::RecordIdentificationHelper
ActionController::Base.helper SimplyHelpful::RecordIdentificationHelper, SimplyHelpful::RecordTagHelper

Rspec's ActionController::Base definition (lib/extensions/action_controller/base.rb):

module ActionController
  class Base
  end
end

ActionController::Base is stripped from all modules included with the

ActionController::Base.send :include, SomeModule

syntax.

Monkey patch to fix

Include in spec/spec_helper.rb, before your Test::Rails::TestCase definition:

module Spec
  module Rails
    class ControllerContext < Rails::Context
        module ContextEvalInstanceMethods
          def setup_extra
            (class << @controller; self; end).class_eval do
              include ControllerInstanceMethods
            end
            @controller.integrate_views! if @integrate_views
            @controller.session = session

            #extend the controller instance with simply_helpful modules
            @controller.extend SimplyHelpful::RecordIdentificationHelper
            @controller.class.helper SimplyHelpful::RecordIdentificationHelper, SimplyHelpful::RecordTagHelper
          end          
        end        
    end
  end
end

Running controller/view specifications using simplyhelpful helper methods, such as domid(), should not raise NoMethodErrors anymore.

Inline RJS in respond_to blocks

This snippet chokes (see comments) when referenced from a controller specification in isolation mode:

 #@language is set in a before_filter
  def create
    @account.supported_languages << @language

    #@language != nil
    respond_to do |format|
      format.js { #@language != nil 
                  render :update do |page|
                    #@language is nil
                    page.replace( dom_id( @language ), :partial => 'shared/admin/language', :locals => { :language => @language  } )
                    page.visual_effect :highlight, dom_id( @language )
                    page.call('Marmalade.Messaging.flash', 'success', "Added language %s" / @language.to_s)
                  end }
      format.xml { @account.languages.to_xml }
    end
  end

Explicitly integrating views with

 integrate_views

properly instantiates a JavaScriptGenerator object, thus preserving controller instance variables within the render :update block.

Any thoughts on this behaviour, or is inline RJS in respond_to blocks generally frowned upon?

Testing different content types with rspec

2006-11-22T03:19:00Z

Rspec ( rather it's dependency ZenTest ) does not currently support content type headers for Controller testing using the Rspec on Rails plugin.

Monkey patch Test::Rails::ControllerTestCase

class Test::Rails::ControllerTestCase

  %w(get put post head delete).each{|m| define_method(m.to_sym) do |*args|
      @request.env['REQUEST_METHOD'] = m.upcase
      @request.env['CONTENT_TYPE'] = Mime::EXTENSION_LOOKUP[(args[2].nil? ? 'html' : args[2].to_s )]
      process( args[0], args[1] || {} )  
    end
  }   

end

Add the above to your Spec helper (spec/spec_helper.rb) after

require 'rspec_on_rails'

and before your Test::Rails::Testcase definition.

Examples

For the sake of saving keystrokes and taking the ability to add your own custom MIME Type extensions in consideration, pass the MIME Type extension (xml, html, js, rss, atom, yaml etc.) as the third argument.Defaults to html if not given.

  get :index, {:locale => 'en'}           #requests HTML content
  get :index, {:locale => 'en'}, :html #requests HTML content
  get :index, {:locale => 'en'}, :xml  #requests XML content
  get :index, {:locale => 'en'}, :js     #requests Javascript content

Using the color-tools gem for theme support

2006-11-21T05:27:00Z

When offering branding options to your customers/users, neither a cusomizable CSS stylesheet nor Liquid is unobtrusive from an integration or usability perspective.

Fall back to your design roots

A standard 3 color palette composed of a

dominant color
interactive color ( complement to your dominant color scheme )
tonic color ( highlights and accents )

requires exactly 3 inputs from your customer/user from which a gradient based scheme can be generated server side for each of the above colors.

Requirements

Austin Ziegler's color-tools gem ( bundled as a dependency when installing pdf-writer ) already implements most of the required functionality.

Install with

sudo gem install color-tools

Parsing hexadecimal (HTML) color values

module Format
  EMAIL = /^[_a-z0-9\+\.\-]+\@[_a-z0-9\-]+\.[_a-z0-9\.\-]+$/i
  HTTP_URI = /^https?:\/\/\S+$/
  HEX_COLOR = /(#)([a-fA-F0-9][a-fA-F0-9])([a-fA-F0-9][a-fA-F0-9])([a-fA-F0-9][a-fA-F0-9])/
end

Format::HEX_COLOR should match the 6 and 7 digit hex color variations eg. #CCCCCC and CCCCCC

The Theme model

class Theme < ActiveRecord::Base
  set_table_name 'themes'

  belongs_to :account
  validates_presence_of :title, :dominant_color, :interactive_color, :tonic_color
  validates_format_of :dominant_color, :with => Format::HEX_COLOR, :message => ' is not a valid HEX color.'.t
  validates_format_of :interactive_color, :with => Format::HEX_COLOR , :message => ' is not a valid HEX color.'.t
  validates_format_of :tonic_color, :with => Format::HEX_COLOR , :message => ' is not a valid HEX color.'.t
  validates_uniqueness_of :title, :scope => 'account_id'
  validates_presence_of :active, :in => 0..1

  serialize :dominant, Array
  serialize :interactive, Array
  serialize :tonic, Array

  delegate :to_s, :to => :title

  class << self 

    def suggested_colors
      Color::RGB::constants.reject{|c| %w(PDF_FORMAT_STR Metallic).include? c }.collect!{|c| eval("Color::RGB::#{c}").html}   
    end

    def color_to_palette( color )
      Color::Palette::MonoContrast.new( Color::RGB.from_html( color ) ).background.to_a.collect!{|c| [c.first, c.last.html] }
    end

  end

  def to_param; "#{self.id}-#{self.title.to_url}" end

  def generate_color_scheme!
    %w(dominant interactive tonic).each do |c|
      self.send("#{c}=".to_sym,  Theme.color_to_palette( self.send("#{c}_color") ) )
    end
  end

  def active?() self.active == 1 end

  def activate!
    return if self.active?
    Theme.transaction do
      self.account.themes.update_all("themes.active = '0'")
      self.update_attribute(:active, '1')
    end
  end

  def before_destroy; !self.active? end 

end

Theme observer

class ThemeObserver < ActiveRecord::Observer

  def before_validation( theme )
    theme.active = 0 unless theme.attribute_present?(:active)
  end

  def before_save( theme ) theme.generate_color_scheme! end

end

Console examples

Shamelessly rip these common colors from color-tools.

>> Theme.suggested_colors
=> ["#cd853f", "#f8f8ff", "#333333", "#ffffe0", "#dc143c", "#a0522d", "#f0e68c", "#e6e6e6", "#48d1cc", "#ff8c00", "#696969", "#ee82ee", "#ffa07a", "#a52a2a", "#da70d6", "#008000", "#808080", "#800000", "#b8860b", "#ff0000", "#708090", "#add8e6", "#f0f8ff", "#ffe4b5", "#8fbc8f", "#9acd32", "#778899", "#d2691e", "#db7093", "#fffaf0", "#ff69b4", "#b3b3b3", "#9370db", "#8b008b", "#f4a460", "#d3d3d3", "#f5f5dc", "#6b8e23", "#9400d3", "#008080", "#4d4d4d", "#00ff00", "#00ffff", "#ffc0cb", "#ffd700", "#c71585", "#9932cc", "#c0c0c0", "#0000ff", "#e6e6fa", "#ffa07a", "#deb887", "#eee8aa", "#1e90ff", "#d02090", "#000000", "#66cdaa", "#a9a9a9", "#bc8f8f", "#adff2f", "#808080", "#ffdead", "#483d8b", "#fffafa", "#f08080", "#faebd7", "#ff7f50", "#ffefd5", "#228b22", "#1a1a1a", "#778899", "#3cb371", "#556b2f", "#2e8b57", "#cd5c5c", "#cccccc", "#6b8e23", "#ff1493", "#d8bfd8", "#90ee90", "#ffe4c4", "#dda0dd", "#daa520", "#4d4d4d", "#32cd32", "#00008b", "#8b0000", "#87ceeb", "#fff0f5", "#191970", "#eee8aa", "#d19275", "#f5deb3", "#20b2aa", "#deb887", "#4169e1", "#808080", "#999999", "#66cdaa", "#006400", "#00ff7f", "#e0ffff", "#00ffff", "#000080", "#2f4f4f", "#ff00ff", "#1a1a1a", "#b0c4de", "#6495ed", "#ffdab9", "#556b2f", "#7b68ee", "#fff5ee", "#4b0082", "#cccccc", "#00bfbf", "#ffebcd", "#d3d3d3", "#ffa500", "#ff6347", "#daa520", "#008b8b", "#faf0e6", "#b0e0e6", "#666666", "#e9967a", "#7cfc00", "#f5fffa", "#6a5acd", "#b22222", "#5f9ea0", "#87cefa", "#98fb98", "#f5f5f5", "#f0fff0", "#a9a9a9", "#0000cd", "#8b4513", "#999999", "#fafad2", "#2f4f4f", "#7fffd4", "#fdf5e6", "#4682b4", "#dcdcdc", "#fff8dc", "#b0c4de", "#ffdab9", "#333333", "#fffff0", "#ff8c00", "#00fa9a", "#fff5ee", "#e6e6e6", "#ffb6c1", "#696969", "#8a2be2", "#ff4500", "#40e0d0", "#ff00ff", "#808080", "#b8860b", "#800080", "#ffffff", "#666666", "#fffacd", "#e9967a", "#ffe4e1", "#708090", "#7fff00", "#8470ff", "#b22222", "#afeeee", "#ffff00", "#ba55d3", "#f0fff0", "#bdb76b", "#fa8072", "#b3b3b3", "#f0ffff", "#fafad2", "#00ced1", "#808000", "#d2b48c"]

Visualizing generated palettes from a saved model.

>> t = Theme.find(1)
>> t.dominant_color
=> "#BBCC44"
>> t.dominant      
=> [[5, "#ecf7f8"], [0, "#44aabb"], [-2, "#33808c"], [1, "#60b7c5"], [-1, "#3a919f"], [2, "#73bfcc"], [-5, "#071113"], [3, "#a2d4dd"], [-4, "#112b2f"], [4, "#d0eaee"], [-3, "#22555e"]]
>> t.interactive_color
=> "#44AABB"
>> t.interactive
=> [[5, "#ecf7f8"], [0, "#44aabb"], [-2, "#33808c"], [1, "#60b7c5"], [-1, "#3a919f"], [2, "#73bfcc"], [-5, "#071113"], [3, "#a2d4dd"], [-4, "#112b2f"], [4, "#d0eaee"], [-3, "#22555e"]]
>> t.tonic_color
=> "#CC6644"
>> t.tonic      
=> [[5, "#faf0ec"], [0, "#cc6644"], [-2, "#994d33"], [1, "#d47d60"], [-1, "#ad573a"], [2, "#d98c73"], [-5, "#140a07"], [3, "#e6b3a2"], [-4, "#331a11"], [4, "#f2d9d0"], [-3, "#663322"]]

Each color palette is an Array, with 11 items.

indexes -1 to -5 : 5 darker shades of the original color
index 0 : the original color
index 1 to 5 : lighter shades of the orginal color

Injecting to the view

There's two easy ways to reflect this in your view ...

Inline style declaration, overriding color values defined in external stylesheets
Use a seperate CSS controller, set content type to text/css and reference in your document head like any other external stylesheets (and the purists are happy too)

Quick what's new here hack

2006-11-20T19:41:00Z

Sometimes you may need to flag a model as being new or recently updated, within in certain time constraint.

Monkey patch/plugin

module Marmalade
  module ActiveRecord
    module Acts
      module Recent

        def self.included( klass )
          super
          klass.extend(ClassMethods)
        end     

        module ClassMethods

          def acts_as_recent( threshold = 5.days )
            class_inheritable_accessor :recent_threshold
            self.recent_threshold = threshold.is_a?( Fixnum ) ? threshold : 5.days
            send :include, Marmalade::ActiveRecord::Acts::Recent::InstanceMethods 
          end
        end

        module InstanceMethods

          def fresh?
            created_at && created_at > Time.now.utc.ago(recent_threshold)
          end

          def recently_updated?
            updated_at && updated_at > Time.now.utc.ago(recent_threshold)  
          end        

        end

      end
    end
  end
end

ActiveRecord::Base.class_eval { include Marmalade::ActiveRecord::Acts::Recent }

Usage

A Fixnum argument is expected.That's on purpose.The ActiveSupport Time extensions makes for clean DSL like reading.

class Booking < ActiveRecord::Base
  acts_as_recent 5.days #3.hours, 1.week etc.

end

Examples

>> b.created_at
=> Mon Nov 20 10:21:23 UTC 2006
>> b.fresh?
=> true
>> b.recently_updated?
=> true
>> b.created_at = Time.now.utc.ago(10.days)
=> Fri Nov 10 19:44:59 UTC 2006
>> b.fresh?
=> false
>> new_bookings, old_bookings = Account.find(2).bookings.by_status(:awaiting_payment).partition{|b| b.fresh?}