CodeGnome Consulting, LTD

Programming - DevOps - Project Management - Information Security

Locking Files With Ruby

| Comments

Ruby’s File#flock doesn’t always work as expected when separate attempts are made to lock a given file. Some Rubyists avoid this method entirely because they assume it’s broken. It’s not broken; it’s simply incorrectly documented.

This post will explain some of the common errors encountered when following the official documentation. It will also provide some concrete examples of how to use File#flock correctly.

Failures and Solutions

File#flock fails in a variety of ways when requesting a lock for a file that has already been locked. Requesting a lock from inside a block releases the lock whenever the block exits, so it doesn’t demonstrate the problem properly.

To demonstrate the problems (and solutions) properly, we need to create a persistent lock that isn’t block-local. In the examples below, the code will make separate attempts to lock the same file in order to expose the issues more clearly.

Timeouts for Exclusive Locks

The File#flock documentation doesn’t fully address the use case of multiple attempts to gain an exclusive lock on a file. This generally results in code that never continues past the second lock attempt.

Infinite Wait with Multiple Exclusive Locks
1
2
3
4
5
6
7
8
# First lock succeeds.
f1 = File.open('foo', File::RDWR|File::CREAT, 0644)
f1.flock(File::LOCK_EX)
# => 0

# This never returns.
f2 = File.open('foo', File::RDWR|File::CREAT, 0644)
f2.flock(File::LOCK_EX)

File#flock doesn’t provide a native mechanism for timing out requests, so the second request will wait indefinitely for an exclusive lock to be obtained. Luckily, another module from Ruby’s standard library can solve this particular problem.

The Timeout module can set a duration for #flock to acquire an exclusive lock. The following example will use a one-millisecond timer to raise Timeout::Error: execution expired, which can then be rescued in whatever way seems appropriate for the application.

Rescue Timeout::Error to Avoid Deadlocks
1
2
3
4
5
6
7
8
9
require 'timeout'

f1 = File.open('foo', File::RDWR|File::CREAT, 0644)
f1.flock(File::LOCK_EX)
# => 0

f2 = File.open('foo', File::RDWR|File::CREAT, 0644)
Timeout::timeout(0.001) { f2.flock(File::LOCK_EX) } rescue nil
# => nil

Returning nil when the timer expires allows the #flock expression to be tested for truth. The inline rescue is therefore particularly appropriate for if/then or case statements, but a more traditional begin/rescue/end block will give you more flexibility in handling the Timeout::Error exception.

Non-Blocking Lock Attempts

Rescuing exceptions is generally slower than evaluating a boolean expression, which is why Ruby supports non-blocking calls to File#flock. However, the documentation for how this works is both incorrect and misleading. The documentation for File#flock says:

Locks or unlocks a file according to locking_constant (a logical or of the values in the table below). Returns false if File::LOCK_NB is specified and the operation would otherwise have blocked.

As a result, following the documentation as written will result in an exception. For example, asking for a non-blocking lock on Linux while some process already holds an exclusive lock results in an invalid argument exception.

Logical OR Raises Exceptions
1
2
3
4
5
6
7
f1 = File.open('foo', File::RDWR|File::CREAT, 0644)
f1.flock(File::LOCK_EX)
# => 0

f2 = File.open('foo', File::RDWR|File::CREAT, 0644)
f2.flock(File::LOCK_NB || File::LOCK_EX)
# => Errno::EINVAL: Invalid argument - foo

Depending on your platform, you may receive other exceptions instead, such as Errno::EBADF. The end result is the same, though: an unclear exception raised from code that conforms to the written documentation for the method.

The problem here is that the File#flock method actually expects a Bitwise OR operator, rather than a Logical OR keyword as defined in parse.y by the tOROP parser token. The correct argument that allows File#flock to return false when an exclusive lock fails is actually File::LOCK_NB|File::LOCK_EX. For example:

Using Bitwise OR for Non-Blocking Requests
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Gain an exclusive lock.
f1 = File.open('foo', File::RDWR|File::CREAT, 0644)
f1.flock(File::LOCK_EX|File::LOCK_NB)
# => 0

# Attempts an exclusive lock and returns immediately. Returns false if
# an exclusive lock was not obtained.
f2 = File.open('foo', File::RDWR|File::CREAT, 0644)
f2.flock(File::LOCK_NB|File::LOCK_EX)
# => false

# Clean up our file handles and release any locks.
f1.close; f2.close
# => nil

This will consistently generate an exclusive lock when available; otherwise, it immediately returns a falsy value without the overhead of raising or rescuing exceptions. This is obviously the way the module is intended to be used, but the documentation could use some clarification and additional examples to make it easier to understand.

Comments