filetype
Here's a handy class to allow retrying a write with flock a set number of times. If it can't flock, it will sleep for a brief random interval and try again. If you have a lot of concurrent writes going on, you can use it to avoid data corruption.
<?php
class SafeWriter
{
// suggested mode 'a' for writing to the end of the file
public static function writeData($path, $mode, $data)
{
$fp = fopen($path, $mode);
$retries = 0;
$max_retries = 100;
if (!$fp) {
// failure
return false;
}
// keep trying to get a lock as long as possible
do {
if ($retries > 0) {
usleep(rand(1, 10000));
}
$retries += 1;
} while (!flock($fp, LOCK_EX) and $retries <= $max_retries);
// couldn't get the lock, give up
if ($retries == $max_retries) {
// failure
return false;
}
// got the lock, write the data
fwrite($fp, "$data\n");
// release the lock
flock($fp, LOCK_UN);
fclose($fp);
// success
return true;
}
}
?>
flock
(PHP 4, PHP 5)
flock — Verrouille le fichier
Description
bool flock
( resource $handle
, int $operation
[, int &$wouldblock
] )
flock() permet de réaliser un système simple de verrous écriture/lecture, qui peut être utilisé sur n'importe quelle plate-forme (Unix et Windows compris).
Le verrou est également levé avec la fonction fclose() (qui est également automatiquement appelée lors de la fin du script).
PHP dispose d'un système complet de verrouillage de fichiers. Tous les programmes qui accèdent au fichier doivent utiliser la même méthode de verrouillage pour qu'il soit efficace.
Liste de paramètres
- handle
-
Un pointeur de fichier ouvert.
- operation
-
operation peut prendre une des valeurs suivantes :
- LOCK_SH pour acquérir un verrou partagé (lecture).
- LOCK_EX pour acquérir un verrou exclusif (écriture).
- LOCK_UN pour libérer un verrou (partagé ou exclusif).
- LOCK_NB si vous voulez que flock() ne se bloque pas durant le verrouillage. (non supporté sous Windows)
- wouldblock
-
Ce troisième argument optionnel est défini à TRUE si le verrou doit bloquer le script (condition d'erreur EWOULDBLOCK).
Valeurs de retour
Cette fonction retourne TRUE en cas de succès, FALSE en cas d'échec.
Historique
| Version | Description |
|---|---|
| 4.0.1 | Les constantes LOCK_XXX ont été ajoutées. Avant, vous deviez utiliser 1 pour LOCK_SH, 2 pour LOCK_EX, 3 pour LOCK_UN et 4 pour LOCK_NB |
Exemples
Exemple #1 Exemple avec flock()
<?php
$fp = fopen("/tmp/lock.txt", "w+");
if (flock($fp, LOCK_EX)) { // pose un verrou exclusif
fwrite($fp, "Écrire dans un fichier\n");
flock($fp, LOCK_UN); // libère le verrou
} else {
echo "Impossible de verrouiller le fichier !";
}
fclose($fp);
?>
Notes
Note: flock() est obligatoire sous Windows.
Note: Comme flock() requiert un pointeur de fichier, vous aurez peut être à utiliser un verrou spécial pour protéger l'accès au fichier que vous voulez tronquer en l'ouvrant en mode d'écriture (avec "w" ou "w+" comme argument de fopen()).
Avertissement
flock() ne fonctionne pas sur NFS ou sur les autres systèmes de fichiers réseaux. Vérifiez la documentation de votre système d'exploitation pour plus de détails.
Sur certains systèmes d'exploitation, flock() est implémenté au niveau processus. Lorsque vous utilisez une API multithread comme ISAPI, vous risquez de ne pas pouvoir avoir confiance en flock() pour protéger vos fichiers contre d'autres scripts PHP qui fonctionnent en parallèle sur d'autres threads du même serveur.
flock() n'est pas supporté sur les vieux systèmes de fichiers comme FAT et ses dérivés, et elle retournera forcément FALSE sous ces environnements (ceci est particulièrement vrai pour les utilisateurs de Windows 98).
add a note
User Contributed Notesflock
tinymountain at nospam dot gmail dot com
31-Jul-2008 10:36
31-Jul-2008 10:36
John dot wellesz at teaser dot fr
15-Apr-2008 02:59
15-Apr-2008 02:59
I just want to add a note about making atomic lock on NFS, there is only two
ways:
- 1 (the most robust but the most complicate) - It's to use link() to create a
hard link to a file you want to lock (on the same FS of course).
(On most NFS implementations, Link() is atomic)
Once you created a hard link (not a symbolic link), with a unique randomly
generated name, call stat() on it and count the number of link (nlink), if there
is only 2 then the file is locked.
If there is more than two you have to unlink() the link you just created and
create a new one with a new unique name (else NFS will use its cache and stat
will return wrong data) then call stat() on the new link and test the number of
links again, repeat this operation until you get the lock.
You have to use usleep() between the link() attempts with a fixed + random
sleep value to avoid dead lock situations (link() and unlink() may be atomic
but not instantaneous)
Also note than when you unlink a file through NFS, if NFS think that the file
is still in use, it will create a .nfs link to this file until it realizes the
file is no longer in use... A wrong timing could generate thousands of those
files and a deadlock situation. Because of this when a deadlock situation
occurs or if your stat() command returns a very high number of links, you have
to look for .nfs file in the same directory you created your links and unlink
all the .nfs file you find (sometimes NFS take its time to remove them)
- 2 (the simplest) - the second method is to use a lock server and lock daemons
on each client that will forward lock request to the server... (this is more
dangerous than the first method because the daemons may be killed...)
Here is for reference the function I created to make atomic locks through NFS
(this function is in production since at least 4 years now), it's just for
reference because it uses many external functions to do its job but you can see
the principle:
http://pastey.net/85793
Daniel Gandra - dgandra at gmail dot com
09-Mar-2008 08:59
09-Mar-2008 08:59
I made some tests with flock () features. I made them because I read on a lot of articles that flock () doesn't work as expected and doesn't block against another processes.
I made the tests with all operation options:
2: shared lock (read mode)
3: exclusive lock (write mode)
5: shared lock (read mode), without blocking the script execution
6: exclusive lock (write mode), without blocking the script execution
All of them worked perfect on windows xp and php 5.2.5
note: when I say "without blocking the script execution" means flock () tries to block and if it can't, it doesn't stay waiting to acquire the lock, the php script continues running, and yeah, in windows even!
I tested with notepad too while the script had locked the file, to see what happened if I made a save on the locked file and... I can't, notepad alerted me: another proccess has the file locked!
The code used is the following:
<?PHP
$content = "";
$input = fopen ("lock_test.txt", "w");
if (flock ($input, 6) == true){
echo "in...<br/>";
sleep (5);
echo "out...<br/>";
flock ($input, 3);
}
else echo "can't acquire block...";
fclose ($input);
//sleep () was used to give myself enough time to switch to another browser window,
//invoke the script again to guarantee parallelism and make the notepad test to see
//if an error occurred with other proccess
?>
admin at ifyowantblood dot de
02-Mar-2008 05:41
02-Mar-2008 05:41
i just read:
>>>>> Note: flock() locks mandatory under Windows.
So flock() really locks the file under windows. this means under windows its better to rely on flock() than to rename a temporary file (because in order to rename a file under windows you have to close the file - and closing it means unlocking it).
admin ifyouwantblood de
24-Dec-2007 04:05
24-Dec-2007 04:05
besides from what the manual says about locking a file opendend in w or w+ and using a special lock file for these cases, you should simply truncate the file yourself with ftruncate() after writing:
<?php
$data='some data';
$handle=fopen('file','r+');
flock($handle,LOCK_EX);
fwrite($handle,$data);
ftruncate($handle,ftell($handle));
flock($handle,LOCK_UN);
fclose($handle);
?>
now the file will have the size of $data without opening the file in w mode but with a lock on the file.
to the previous writers jpriebe and mallory:
of course the lock is lost in this case, but thats simply because the file is closed by PHP. and closing the file means unlocking it (same as when you use fclose() yourself).
jpriebe at cbcnewmedia dot com
19-Dec-2007 06:18
19-Dec-2007 06:18
mallory.dessaintes' message leads to another challenge with flock() -- if the file handle goes out of scope, the locking mechanism doesn't work.
Imagine you have a long-running command-line PHP script. You schedule it to run via cron every minute, but you only want one copy at a time to run.
You might implement a function like this:
<?php
function get_lock ()
{
$fp = @fopen ('/tmp/myscript.lock', 'w+');
if (!$fp)
{
return false;
}
if (flock ($fp, LOCK_EX + LOCK_NB))
{
return true;
}
return false;
}
?>
But when the function returns, the $fp variable goes out of scope, and the lock is broken. You need to make your file handle variable global like this:
<?php
function get_lock ()
{
global $g_lock_fp;
$g_lock_fp = @fopen ('/tmp/myscript.lock', 'w+');
if (!$g_lock_fp)
{
return false;
}
if (flock ($g_lock_fp, LOCK_EX + LOCK_NB))
{
return true;
}
return false;
}
?>
mallory dot dessaintes at gmail dot com
19-Dec-2007 04:56
19-Dec-2007 04:56
I have noticed that if you change the value of your fopen ressource, the lock is working no longer..
<?php
$fo = fopen('lockfile.txt','a');
flock($fo,LOCK_EX);
$fo = '';
// Lock is disable
?>
candide at idees-et-solutions dot fr
07-Dec-2007 12:48
07-Dec-2007 12:48
Just a comment about the last method to lock files using filemtime().
What if filemtime($fp[1]) == $fp[3] because somebody modified the file less than 1s after the value of $fp[3] was picked up?
Then this modification will be lost...?
This system to lock files is made to prevent problems when two modifications are so close that they can interfere, so the case "less than 1s" will often happen?
However, lose some modifications is better than spoil all the file...
info at NOSPAMPLEASE dot c-eagle dot com
09-Oct-2007 03:55
09-Oct-2007 03:55
If there is a file that´s excessively being rewritten by many different users, you´ll note that two almost-simultaneously accesses on that file could interfere with each other. For example if there´s a chat history containing only the last 25 chat lines. Now adding a line also means deleting the very first one. So while that whole writing is happening, another user might also add a line, reading the file, which, at this point, is incomplete, because it´s just being rewritten. The second user would then rewrite an incomplete file and add its line to it, meaning: you just got yourself some data loss!
If flock() was working at all, that might be the key to not let those interferences happen - but flock() mostly won´t work as expected (at least that´s my experience on any linux webserver I´ve tried), and writing own file-locking-functions comes with a lot of possible issues that would finally result in corrupted files. Even though it´s very unlikely, it´s not impossible and has happened to me already.
So I came up with another solution for the file-interference-problem:
1. A file that´s to be accessed will first be copied to a temp-file directory and its last filemtime() is being stored in a PHP-variable. The temp-file gets a random filename, ensuring no other process is able to interfere with this particular temp-file.
2. When the temp-file has been changed/rewritten/whatever, there´ll be a check whether the filemtime() of the original file has been changed since we copied it into our temp-directory.
2.1. If filemtime() is still the same, the temp-file will just be renamed/moved to the original filename, ensuring the original file is never in a temporary state - only the complete previous state or the complete new state.
2.2. But if filemtime() has been changed while our PHP-process wanted to change its file, the temp-file will just be deleted and our new PHP-fileclose-function will return a FALSE, enabling whatever called that function to do it again (ie. upto 5 times, until it returns TRUE).
These are the functions I´ve written for that purpose:
<?php
$dir_fileopen = "../AN/INTERNAL/DIRECTORY/fileopen";
function randomid() {
return time().substr(md5(microtime()), 0, rand(5, 12));
}
function cfopen($filename, $mode, $overwriteanyway = false) {
global $dir_fileopen;
clearstatcache();
do {
$id = md5(randomid(rand(), TRUE));
$tempfilename = $dir_fileopen."/".$id.md5($filename);
} while(file_exists($tempfilename));
if (file_exists($filename)) {
$newfile = false;
copy($filename, $tempfilename);
}else{
$newfile = true;
}
$fp = fopen($tempfilename, $mode);
return $fp ? array($fp, $filename, $id, @filemtime($filename), $newfile, $overwriteanyway) : false;
}
function cfwrite($fp,$string) { return fwrite($fp[0], $string); }
function cfclose($fp, $debug = "off") {
global $dir_fileopen;
$success = fclose($fp[0]);
clearstatcache();
$tempfilename = $dir_fileopen."/".$fp[2].md5($fp[1]);
if ((@filemtime($fp[1]) == $fp[3]) or ($fp[4]==true and !file_exists($fp[1])) or $fp[5]==true) {
rename($tempfilename, $fp[1]);
}else{
unlink($tempfilename);
if ($debug != "off") echo "While writing, another process accessed $fp[1]. To ensure file-integrity, your changes were rejected.";
$success = false;
}
return $success;
}
?>
$overwriteanyway, one of the parameters for cfopen(), means: If cfclose() is used and the original file has changed, this script won´t care and still overwrite the original file with the new temp file. Anyway there won´t be any writing-interference between two PHP processes, assuming there can be no absolute simultaneousness between two (or more) processes.
Antti Haapala
06-Oct-2007 01:30
06-Oct-2007 01:30
Further information on flock: The system is not restarted if a signal is delivered to the process, so flock will happily return false in case of SIGALRM, SIGFPE or something else.
Antti Haapala
06-Oct-2007 12:41
06-Oct-2007 12:41
The supplied documentation is vague, ambiguous and lacking, and the user comments contain erroneous information! The flock function follows the semantics of the Unix system call bearing the same name. Flock utilizes ADVISORY locking only; that is, other processes may ignore the lock completely; it only affects those that call the flock call.
LOCK_SH means SHARED LOCK. Any number of processes MAY HAVE A SHARED LOCK simultaneously. It is commonly called a reader lock.
LOCK_EX means EXCLUSIVE LOCK. Only a single process may possess an exclusive lock to a given file at a time.
If the file has been LOCKED with LOCK_SH in another process, flock with LOCK_SH will SUCCEED. flock with LOCK_EX will BLOCK UNTIL ALL READER LOCKS HAVE BEEN RELEASED.
If the file has been locked with LOCK_EX in another process, the CALL WILL BLOCK UNTIL ALL OTHER LOCKS have been released.
If however, you call flock on a file on which you possess the lock, it will try to change it. So: flock(LOCK_EX) followed by flock(LOCK_SH) will get you a SHARED lock, not "read-write" lock.
administrator at proxy-list dot org
30-Apr-2007 02:55
30-Apr-2007 02:55
The bug fix in my last post. During test I have discovered that if you have locked file for writing (LOCK_EX) both read and write will not be accessible from other scripts. In case of read locking (LOCK_SH) only writing will not be accessible for other PHP scripts but they will be able to read file simultaneously.
administrator at proxy-list dot org
30-Apr-2007 02:30
30-Apr-2007 02:30
Hello,
I want to give an example how to lock file with two or more flags (for example reading and writing). IMPORTANT: each locking should be done separately, the correct way of using flock() is:
<?php
flock($fp, LOCK_EX);
flock($fp, LOCK_SH);
?>
and NOT like these:
<?php
flock($fp, LOCK_EX and LOCK_SH);
flock($fp, LOCK_EX or LOCK_SH);
flock($fp, LOCK_EX + LOCK_SH);
?>
Furthermore if someone has not pay attention to function’s description - flock does not lock any file in the right way. The file is still accessible for reading/writing, in other words these functions: file(), file_get_contents() and even fopen($file, ‘r’) will ignore the lock.
I think PHP mechanism works something like this: as soon as file lock was successful, the function flock() writes somewhere (in its own “DB” for example) that file handle is locked with some flag and nothing more. It is up to the developer to check if file is locked or not before doing any operations.
Hope this post makes clear the flock() function’s working principles.
Regards
Vitali Simsive
LemonJuice
09-Apr-2007 09:37
09-Apr-2007 09:37
Hi,
The discussions below address flock() in the context of managing integrity of file contents as well as the context of using flock() in combination with a dummy file to generally establish agreement on the access state of some other object. The following addresses the latter.
I use this as a replacement for LOCK TABLES because during some transactional update statements I require the contents of other tables to freeze and transactions and tablelocks don't mix in mySQL / InnoDB.
<?php
class ReadWriteLock
{
const LOCK_PATH = "locks";
public static function Aquire($ID, $LockType = LOCK_SH, $WouldBlock = TRUE)
{
// Make sure the file exists and we have it opened.
// We don't care about writing to the file. We just need a file reference that flock() can work on.
// Also, on an OS level all thread's are sharing this file. We don't do access control in relation to this file.
// So let's assume first that it already exists.
$FileName = self::LOCK_PATH."/lock_$ID.lck";
if(($Resource = @fopen($FileName, "r")) === FALSE)
// Ok, so this is the first time a thread acquires a lock to this $ID. Let's create the file.
if(($Resource = @fopen($FileName, "w")) === FALSE)
{
// Ok, perhaps some thread created it between the two ifs. This class does not delete the file so it should now be there.
if(($Resource = @fopen($FileName, "r")) === FALSE)
return FALSE;
}
else
{
// #REF 1
// Ok, it exists now. Just for solidarity and prevention of whatever OS hickups we can possibly have
// I want this thread to open it in r mode too.
if(fclose($Resource) === FALSE)
return FALSE;
if(($Resource = @fopen($FileName, "r")) === FALSE)
return FALSE;
}
// And this is really where the locking takes place.
if(flock($Resource, $LockType, $WouldBlock)) return $Resource;
fclose($Resource);
return FALSE;
}
public static function Release($Resource)
{
if(fclose($Resource)) return TRUE;
return FALSE;
}
}
// Entering critical section
if(($Lock = ReadWriteLock::Acquire("metadata")) === FALSE)
die("Failed to either create or acquire the lock.")
// Construct SQL statements from meta tables
// Execute constructed SQL statements agains data tables
// Leaving critical section
if((ReadWriteLock::Release($Lock)) === FALSE)
die("Failed to release the lock.");
?>
A few notes about this:
- As you can see, this just creates 'a' file to use as a reference for flock() to work on. My assumption here is that the operating system uses semaphores internally to implement Flock(). Of that, I am not sure however and I would appreciate any validation from an expert.
- The problem of the existence of the lockfile is solved by simply not deleting them. Given that they are all 0-byte files in a specified folder and that they are only of a limited amount makes it something that works for my solution. Alternatively you could touch() the lockfile upon a succesful flock() and use a cron job to delete any files that have not been touched since, say, a day (or at least for the duration of the session timeout setting of your webserver). That would introduce a race condition for access on the actual file though which I prefer to exclude from the above.
- flock() implements a low priority exclusive lock. This means that once the resource is locked in a shared mode, exclusive locks may be delayed indefinately if (and only if) a continuous abundance of shared lock requests come in so that every thread releases his shared lock after another thread has already gained shared access. For me, this is an ussue and I would appreciate any references to establish a high priority exclusive lock.
Good luck,
Juice
...Tastes like more!
Will Reiher
28-Mar-2007 02:52
28-Mar-2007 02:52
I've been testing a few custom file access functions but I always liked the simplicity of file_get_contents(). Of course it doesn't seem to respect any file locks created with flock(). I created the function below to wrap around file_get_contents() so it can support locked files. It's an odd way of doing it but it works for me.
function flock_get_contents($filename){
$return = FALSE;
if(is_string($filename) && !empty($filename)){
if(is_readable($filename)){
if($handle = @fopen($filename, 'r')){
while(!$return){
if(flock($handle, LOCK_SH)){
if($return = file_get_contents($filename)){
flock($handle, LOCK_UN);
}
}
}
fclose($handle);
}
}
}
return $return;
}
Corey
07-Feb-2007 06:01
07-Feb-2007 06:01
In response to korostel:
Using files for things like counters is a very bad idea...especially when you're requiring scripts to wait until the file is available before accessing the file.
You should use a database like MySQL or PostgreSQL since they are much better performance-wise (especially MySQL).
If you need the data to be in a file for some reason you can always use a cron job that reads the current data from your database and then writes it to the file.
korostel at newmail dot ru
29-Jan-2007 12:22
29-Jan-2007 12:22
In order to prevent access to some counter file we can use another file as a flag instead of flock().
We change flag's file mode to '400' each time we want to change the counter, and set it back to '600' at the end.
If the mode of the flag's file was already changed by another process, we make delayed loop till its mode is set to writable again.
The page is refreshing every second so the results can be seen by opening it in two or three browser windows. I tested the script meny times and still my counter is safe.
Hope this will help.
--------------START------------------
<?
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");
header("Cache-Control: no-cache, must-revalidate");
header("Pragma: no-cache");
header("Last-Modified: ".Date("D, d M Y H:i:s")." GMT");
define('COUNT_FILE','count.txt'); //Our counter file
define('LOCK_FILE','lock.txt'); //Our lock file.
function read_write () {
//making our lock file nonwritable
chmod(LOCK_FILE, 0400);
//reading
clearstatcache();
$fpr = fopen(COUNT_FILE, "r");
$count=fread ($fpr, filesize (COUNT_FILE));
fclose($fpr);
$count++;
//writing
$fpw = fopen(COUNT_FILE, "w");
fwrite($fpw,$count);
fclose($fpw);
//pause the script just to see how it works
sleep(1);
//making our lock file writable again
chmod(LOCK_FILE, 0600);
return $count;
}
?>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head><title>Untitled</title><meta http-equiv="Refresh" content="1"></head><body>
<?
clearstatcache();
//Check the mode of our lock file
if(is_writable(LOCK_FILE)) {
print 'is unlocked <br>';
//make changes and show the results
print 'changed to '.read_write().'<br>';
}else{
print 'is locked: ';
//make loop
while(!is_writable(LOCK_FILE)) {
print 'pause... ';
usleep(rand(5,999));
}
//make changes and show the results
print '<br>changed at last to '.read_write().'<br>';
}?>
</body></html>
--------------END------------------
llinear
27-Jan-2007 01:36
27-Jan-2007 01:36
There are few instances where several instances of a server script need write access to the same file. But it is not mission critical (you would not be using PHP) that the write succeeds, and therefore not worth making a script wait. If its just for a counter, then you can afford to skip a client from time to time... Bots already screwed your count
georgeskesseler at yahoo point com
28-Dec-2006 05:55
28-Dec-2006 05:55
If you do some testing opening several tabs in firefox on the same URL you may find that LOCK_NB seems not to get respected and $wouldblock always is 0. Apparently this is a firefox problem. Try 2 different browsers or 2 separate wget, then there is no problem.
http://bugs.php.net/bug.php?id=36824
I also confirm that you have to test the value of $wouldblock, not the return value of flock() if using LOCK_NB on linux (debian/ubuntu 2.6.15-26-386)
jerry at gh33 dot org
14-Aug-2006 06:30
14-Aug-2006 06:30
Indeed, flock() will not work reliably when the underlying filesystem is NFS. The proper way to perform file locking, in this case, would be to use PHP's link() function. From the Linux man page of open():
O_EXCL When used with O_CREAT, if the file already exists it is an
error and the open will fail. In this context, a symbolic link
exists, regardless of where its points to. O_EXCL is broken on
NFS file systems, programs which rely on it for performing lock-
ing tasks will contain a race condition. The solution for per-
forming atomic file locking using a lockfile is to create a
unique file on the same fs (e.g., incorporating hostname and
pid), use link(2) to make a link to the lockfile. If link()
returns 0, the lock is successful. Otherwise, use stat(2) on
the unique file to check if its link count has increased to 2,
in which case the lock is also successful.
wildcat at brightNOSPAMpages dot net
08-Apr-2006 08:48
08-Apr-2006 08:48
I wrote the following function to make a script wait in a queue of instances accessing the same shared resource and then lock it, as a part of a lock-access-release advisory mechanism based on locking directories.
It works on my web site (also passes the loop test proposed by dranger below). Might show unpredicted behaviors on other systems, under conditions I didn't think at or produce, expecially when mkdir atomicity is not guaranteed.
The whole script is too long for posting, but everyone interested in looking for eventual bugs, testing, torturing and using it, can download code and documentation from http://brightpages.net/doc > QLCK.
Hope it is useful.
<?php
function qLock($lockSuffix, $tmpDir, $maxWait, $padLength) {
if ($tmpDir == '' or !is_dir($tmpDir)) $tmpDir = '.';
$maxWait = (int) $maxWait; if ($maxWait <= 0) $maxWait = 60;
$padLength = (int) $padLength; if ($padLength <= 0) $padLength = 6;
if ($lockSuffix == '') {
return false;
} else {
$currentdir = getcwd();
chdir($tmpDir); // relative to the current directory
$n = 0;
$lockdirs = glob('*-*.'.$lockSuffix, GLOB_ONLYDIR) or array();
foreach($lockdirs as $lockdir) {
list($m, ) = explode('-', $lockdir);
$m = intval($m);
if ($m > $n) $n = $m + 1;
}
$npad = str_pad((string) $n, $padLength, '0', STR_PAD_LEFT);
while (glob($npad.'-*.'.$lockSuffix, GLOB_ONLYDIR)) {
$n++;
$npad = str_pad((string) $n, $padLength, '0', STR_PAD_LEFT);
}
$lockname = $npad.'-'.md5(uniqid(rand(), true)).'.'.$lockSuffix;
if (@mkdir($lockname)) {
$nlockdirs = glob($npad.'-*.'.$lockSuffix, GLOB_ONLYDIR) or array();
if (count($nlockdirs) != 1) {
@rmdir($lockname);
return false;
} else {
$prelockdirs = array();
$lockdirs = glob('*-*.'.$lockSuffix, GLOB_ONLYDIR) or array();
foreach($lockdirs as $lockdir) {
list($m, ) = explode('-', $lockdir);
$m = intval($m);
if ($m < $n) $prelockdirs[] = $lockdir;
}
$t0 = time(); $t = 0;
while (count($prelockdirs) != 0 and $t < $maxWait) {
$prelockdirs = array();
$lockdirs = glob('*-*.'.$lockSuffix, GLOB_ONLYDIR) or array();
foreach($lockdirs as $lockdir) {
list($m, ) = explode('-', $lockdir);
$m = intval($m);
if ($m < $n) $prelockdirs[] = $lockdir;
}
$t = time() - $t0;
}
if (count($prelockdirs) != 0) {
@rmdir($lockname);
return false;
} else {
return $lockname;
}
}
} else {
@rmdir($lockname);
return false;
}
chdir($currentdir);
}
}
?>
christian dot wessels at web dot de
07-Apr-2006 11:41
07-Apr-2006 11:41
dranger at export dash japan dot com is right, i once had the problem of a race condition in a database action because the script was very cpu-intensive so i looked around for some kind of mutex.
I finally came up to this which uses no files (and no flock) and survived all of my tests (on linux):
<?php
function generate_shmkey ($string)
{
// Private: output unique integer based on $string
return abs(crc32($string));
}
function vmutex_lock ($lockId, $maxAcquire = 1)
{
// Public: take cover of semaphores and lock when possible
global $shm_handle;
$shm_key = generate_shmkey($lockId);
if ($shm_handle = sem_get($shm_key, $maxAcquire))
{
if (sem_acquire($shm_handle))
{
return true;
}
}
return false;
}
function vmutex_unlock ($lockId)
{
// Public: remove lock
global $shm_handle;
return sem_release($shm_handle);
}
?>
heshan at sjtu dot edu dot cn
01-Apr-2006 08:08
01-Apr-2006 08:08
dranger at export dash japan dot com 's suggestion is correct and useful. My purpose is just preventing any other process from disturbing the current process which is rewriting a "cache" file, and flock() itself can help me to achieve this.
$fp = fopen( "test.txt", "ab" );
if( $fp && flock( $fp, LOCK_EX ) ) {
ftruncate( $fp, 0 );
fwrite( $fp, $part_one );
sleep( 10 ); // for test purpose, assume the whole writing process takes 10 seconds
fwrite( $fp, $part_two );
fclose( $fp );
}
it works well, I open 3 browser windows to request this page, and the third one actually takes more than 20( about 26 ) seconds to finish. and the content of cache.txt is correct and up-to-time.
tested under:
1, windows, apache2, php4.4.1
2, freebsd, apache, php4.2.3
dranger at export dash japan dot com
06-Mar-2006 06:45
06-Mar-2006 06:45
Re: Niels Jaeckel
The code posted unfortunately does NOT work in race conditions, because even though it is improbable, there is still a chance the 'mutex' will fail and you will encounter a situation where two programs have a 'lock' for the same file at the same time. It is improbable, but in a busy system with perhaps many processes trying to open one file or perhaps two processes opening the file several times, the probability rises and you will most certianly get strange bugs at strange times.
Even though it looks quite improbable, there are two problems:
1) The extra ifs do not add any extra precautions. You might as well usleep for a few microseconds because only the last one matters.
2) Even though it looks like a single atomic command, if you look at the PHP source code, touch is relatively complicated. It gets the time from the computer, checks the base directory, etc.,etc.
This means that there will be times that this lock fails, no matter how improbable. What makes a race condition so nasty is that it happens only once every so often, making it nearly impossible to debug and really annoying.
*** The only safe way to implement locks is with flock or some other locking mechanism *outside* of PHP. ***
(caveat: there may be an alternate locking mechanism in PHP i don't know about, but you cannot make your own)
To put this to rest, make a PHP script using the functions described in the previous post. Then, add this code to the file:
<?php
/*** test.php ***/
// lock and unlock function defs go here
while(1) {
if(lock("test")) {
$f=fopen("importantfile", "w") or die;
$pid=getmypid();
$string="Important Information! From $pid";
fwrite($f, $string);
fclose($f);
$check=file_get_contents("importantfile");
if($check != $string) {
echo "THIS LOCK FAILED!\\n";
}
unlock("test");
}
}
?>
Then run this script from the command line - it will loop forever happily. Then, *while that script is running*, run the same script again while the first one is still going. In a UNIX environment you can do this by typing:
> php test.php &
> php test.php &
You will probably see this:
Warning: unlink(test.lock): No such file or directory in lock.php on line 29
THIS LOCK FAILED!
Warning: touch(): Utime failed: No such file or directory in lock.php on line 19
a lot of times.
This means the lock has failed :)
In fact, if you ever think you have invented a clever way to lock a file, test it first in this while loop. Just replace lock and unlock with your function and rearrange the code so it makes sense. Then run it and see if it fails.
Note that flock() passes this test beautifully.
Niels Jaeckel
03-Mar-2006 04:43
03-Mar-2006 04:43
hi dranger at export dash japan dot com,
you are right with your thoughts about process-switching. But I think the following kind of code would work in race-conditions, because it is very improbably that the CPU switches after just ONE statement:
<?php
// here my own lock-function WITHOUT flock()
// you can call it a mutex...
// lock ... compare to P(mutex)
function lock($filename) {
$filename .= '.lock';
while (true) {
if (! file_exists($filename)) {
if (! file_exists($filename)) {
if (! file_exists($filename)) {
return touch($filename);
}
}
}
}
}
// unlock ... compare to V(mutex)
function unlock($filename) {
unlink($filename . '.lock');
}
?>
If you want to solve the reader/writer problem (many readers, just one exclusive writer) you can add some flock() - code instead of touch(...)
hth
dranger at export dash japan dot com
22-Feb-2006 03:19
22-Feb-2006 03:19
jbr at ya-right dot com has the wrong idea of what flock is supposed to do.
flock is *supposed* to "hang" until the other lock is released. This is the intended behavior on ALL systems, not just Windows.
If this behavior is not acceptable, flock already has a mechanism to deal with this: LOCK_NB. Just add LOCK_NB to the second argument, and flock will not "hang." For example:
<?php
flock($fp, LOCK_EX+LOCK_NB, $wouldblock)
?>
and flock will return TRUE or FALSE immediately, setting $wouldblock to 1 if the call would have blocked.
You DO NOT need to muck about with while loops, usleep, "clever" md5 + time hashing tricks, random numbers, or any of this nonsense. These techniques defeat the entire point of using flock in the first place.
What's worse about jbr's code is that it introduces a bug that flock was meant to fix in the first place! It's called a "race condition" - here's an example:
1. Program A checks that file "fakelock" exists. It doesn't.
2. Processor switches to Program B.
3. Program B checks that file "fakelock" exists. It doesn't.
4. Program A writes its unique key, "A" to the file, and checks that its key is correct. It is.
5. Program B writes its unique key "B" to the file, and because its file pointer was pointing at the beginning of the file, it overwrites "A" with "B". It checks that its key is correct. It is.
6. Now Program A and Program B can both write to file "$fl_file.", and both overwrite the file that was to be protected. There are many other situations in which this code will fail as well.
To the best of my knowledge, YOU CANNOT MAKE YOUR OWN FLOCK USING JUST PHP NO MATTER HOW HARD YOU TRY. YOU MUST USE FLOCK.
If you have Windows ISAPI, you may have trouble with flock working correctly because it is multithreaded, but this kind of fix will not correct the situation.
jbr at ya-right dot com
19-Feb-2006 04:17
19-Feb-2006 04:17
On Windows ISAPI testing a file or directory with flock does not return
TRUE OR FALSE, it just results in a system hang until the prior flock
called has been released. So it is sometimes better to create a fake
locking system that gives you complete control of the IO handle so you
can exist or move on if the lock is not released at a certain number
of tries or time, to get at the file!
// PHP 5 example
<?
$file = './log.txt'; // the file we are writing to
$data = "add this\r\n"; // data to add
$write_type = 'a'; // append to the file
$lock_name = './lock'; // the fake locking file name
$lock_tries = 10; // the number of times to try and open the file
// usage
if ( fake_lock ( $file, $data, $write_type, $lock_name, $lock_tries ) === true )
{
echo 'data written to file';
}
else
{
echo 'could not write data to file!';
}
function fake_lock ( $fl_file, $fl_data, $fl_write, $fl_name, $fl_try )
{
$fl_done = false;
$fl_count = 0;
$fl_key = md5 ( uniqid ( microtime () ) );
do
{
$fl_count++;
if ( ! file_exists ( $fl_name ) )
{
file_put_contents ( $fl_name, $fl_key );
$fl_test = file_get_contents ( $fl_name );
if ( $fl_test == $fl_key )
{
$fl_done = true;
}
}
if ( ! $fl_done )
{
if ( $fl_count == $fl_try )
{
return ( false );
}
else
{
usleep ( 10000 );
}
}
} while ( ! $fl_done );
$io = fopen ( $fl_file, $fl_write );
fputs ( $io, $fl_data );
fclose ( $io );
unlink ( $fl_name );
return ( true );
}
?>
dranger AT export dash japan dot com
17-Feb-2006 07:03
17-Feb-2006 07:03
Also note that if you want to truncate a file, but make sure it's locked first, you DON'T need to use a separate lock file like the directions say. Use this instead:
<?php
$f=fopen("file", "r+");
flock($f, LOCK_EX) or die("Error! cant lock!");
ftruncate($f, 0);
fwrite($f, $stuff);
fclose($f);
?>
But if you open a file with "w" or "w+" you WILL blow it away before you can lock it.
dranger AT export dash japan dot com
17-Feb-2006 06:50
17-Feb-2006 06:50
You should never have to write code like this:
<?php
while(!($canWrite=flock($f, LOCK_EX))) {
usleep(2000);
}
// go ahead and use file
fwrite(...);
fclose($f);
?>
This absolutely defeats the purpose of OS-supported locks, and what's worse is if you specify LOCK_NB, it eats up your CPU time.
The following code will work the same way:
<?php
// this will automatically block, that is, PAUSE until the lock is released
flock($f, LOCK_EX) or die("Error getting lock!");
// go ahead and use file
fwrite(...);
fclose($f);
?>
I'm not sure what these people creating spin/sleep loops are doing.
marc dot vanwoerkom at fernuni-hagen dot de
15-Feb-2006 11:00
15-Feb-2006 11:00
I ran into a loop because I just checked for true (= you got the lock) as return value of flock() and tried again when I got a false.
function naive_wait_for_file($fp) {
while (true) {
if (flock($fp, LOCK_EX)) {
return;
}
$k = rand(0, 20);
usleep(round($k * 10000)); # k * 10ms
}
}
Unfortunately in one case the $fp I put in was invalid, so I always got false and got stuck.
Lesson: check if your $fp is valid before entering the loop, or look closer if you get a false.
function wait_for_file($fp) {
if ($fp === false) {
return;
}
while (true) {
if (flock($fp, LOCK_EX)) {
return;
}
$k = rand(0, 20);
usleep(round($k * 10000)); # k * 10ms
}
}
Ivan
04-Feb-2006 09:48
04-Feb-2006 09:48
damasta at onwebworx dot net said that it is impossible to delete a lock file inside of a register_shutdown_function callback. I found a way to do that. You just have to use an absolute path to the file:
<?php
function shutdown_callback() {
unlink(dirname($_SERVER['SCRIPT_FILENAME']) . "/lock.lock"); //otherwise it wouldn't work
echo "<h1>Terminating</h1>\n";
}
?>
Funny that echo works too, although PHP Manual says it shouldn't work :)
arne at bukkie dot nl
19-Jan-2006 11:27
19-Jan-2006 11:27
An addition to administrator at proxy-list dot org snippet below.
To prevent unnecessary waiting check if the lock is obtained, if so skip the usleep():
$fp = fopen($logFileName, 'a');
$canWrite = false;
//Waiting until file will be locked for writing
while (!$canWrite) {
$canWrite = flock($fp, LOCK_EX);
// If lock not obtained sleep for 0 - 2000 miliseconds, to avoid colision
if( !$canWrite ) {
$miliSeconds = rand(0, 20); //1 u = 100 miliseconds
usleep(round($miliSeconds*100000));
}
}
//file was locked so now we can store information
fwrite($fp, $toSave);
fclose($fp);
12-Jan-2006 11:51
If you use a database, you can also create timeout locks.
administrator at proxy-list dot org
29-Dec-2005 05:12
29-Dec-2005 05:12
Hello guys,
I want to shear one good trick, it was no invented by me but it is very useful with flock. This technology was used by team who invent Ethernet as my tutor Pitter Timothy teach me. I want to say thank you.
If you have a lot of scripts about 1000 which possible try to write something in file you need to lock file before starting writing. So you should use something like this:
$fp = fopen($logFileName, 'a');
$canWrite = false;
//Waiting until file will be locked for writing
while (!$canWrite) {
$canWrite = flock($fp, LOCK_EX);
}
//file was locked so now we can store information
fwrite($fp, $toSave);
fclose($fp);
but during testing I have find out what some times script have to many collisions, and during 10 seconds can not write anything. It happened because some scripts try simultaneously. If file was busy they all will wait same time. So I use the same technology like guys who invent first simple Ethernet use in case of package collision. I put random millisecond sleep. You can not imagine but script start working 3 times quicker!
$fp = fopen($logFileName, 'a');
$canWrite = false;
//Waiting until file will be locked for writing
while (!$canWrite) {
$canWrite = flock($fp, LOCK_EX);
//Sleep for 0 - 2000 miliseconds, to avoid colision
$miliSeconds = rand(0, 20); //1 u = 100 miliseconds
usleep(round($miliSeconds*100000));
}
//file was locked so now we can store information
fwrite($fp, $toSave);
fclose($fp);
By the way I have no idea which diapason is better, but 0 1000 is not enough.
Hope it will help somebody
Best regards in your projects
Vitali Simsive
j t [atthedomain] imen&org&uk
15-Nov-2005 03:54
15-Nov-2005 03:54
fackelkind honorsociety de wrote earlier a function to help with lock - but they should have used an incremental backoff solution - instead of sleeping for 30, w ould it not be nicer to sleep for 30 * n where n is the loop itteration?
relsqui at chiliahedron dot com
10-Nov-2005 12:43
10-Nov-2005 12:43
Well, I can claim I wrote this as a workaround for earlier versions of PHP, but in reality I just didn't think to look up flock() until after I'd written it. However, if you do happen to have an earlier version of PHP, these might come in handy. Enjoy.
<?php
// Lock a file, timing out if it takes too long.
function lock ($lock, $tries) {
$lock0 = ".{$lock}0";
$lock1 = ".{$lock}1";
for ($i=0; $i<$tries; $i++) {
if (!is_file($lock0)) {
touch($lock0);
if (!is_file($lock1)) {
touch($lock1);
return 1;
}
}
usleep(100);
}
return 0;
}
// Unlock a file.
function unlock ($lock) {
unlink(".{$lock}1");
unlink(".{$lock}0");
}
// Usage example.
$filename = "somefile";
$data = "stuff and things\n";
$tries = 10;
if (lock($filename, $tries)) {
$h = fopen($filename, "a") or die();
fwrite($h, $data);
fclose($h);
unlock($filename);
} else {
die("Failed to lock $filename after " . ($tries*100) . " milliseconds!";
}
?>
kit dot lester at lycos dot co uk
18-Sep-2005 01:49
18-Sep-2005 01:49
Amazing! these discussions are so similar to those back in the late '60s and early '70s about how to achieve file synchronization - especially across not-very-compatible platfoms.
That was the big debate that led us to create databases, rollback and all, for exactly the same reasons.
The only big difference between then and now is that these days we have the internet, whereas back then the discussion was in the letters columns of the computing journals. [E-mail only started to be available in the late 70s - and messy to use via gateways between different national nets.]
PHP's flock() is a nice little hack for simple synchronization situations: trying to use it in big situations won't reliably work. If you need a bit of resilience in the application, you have to provide it yourself - which could be as simple as occasional manual backups of the files that could become corrupted.
The mkdir/etc trick is a slightly bigger hack: but horridly dependant on atomicity, which is not guaranteed on guess-which-corporation's crummy operating systems. Consider switching to Unix/Linux/Mac.
If you have a big data-handling problem needing high automatic resilience, then you need a database: sadly, no-one's come up with a better answer in 40 years of trying. If PHP+MySQL is too slow, you need either one of the faster expensive databases, or servelets.
Or wait for CPUs to get fast enough for you.
Or rethink the problem you're trying to solve [which is what I've done many many times over the decades: the resulting application is invariably better than anything I'd have achieved with the original poor problem-analysis.]
----------------
It's depressing how people will struggle with inventing a square wheel without checking whether it was invented before. "Those who know no history are condemned to repeat it." [Carlos Casteneda - but similar thoughts were expressed by Churchill, Napolean, Alexander of Macedon, and many others.]
vlad at NOTNEEDED easysoft dot ee
08-Sep-2005 11:52
08-Sep-2005 11:52
I was searching for the idea to get flock() to work without data loss, and unfortunately all i read here was not helpful to me, after some hard testing i still get some data lost or corrupted.
So my suggestion is how to make a really data safe mechanism is to organize something like simple multithreaded queue daemon, that will listen for connections and wait for data to be sent and push it to some queue. Then process data from the queue and store it to file/sql or whatever we need.
Another words: we get data queued and then processed and stored without any flock(); Also this gives us possibility to have centralized data sysytem (in case we need to collect data from different remote servers).
Sure this will be not so easy as flock(), so people who count on proffessionalism and not luck, may find this helpful.
Thnx.
fackelkind honorsociety de
16-Aug-2005 04:58
16-Aug-2005 04:58
If found a handy way for myself, to handle a filelock on linux and windows:
<?php
function &file_lock (&$handle, $lock = true){
while (false === flock ($handle, ($lock == false) ? LOCK_UN : LOCK_EX + LOCK_NB))
usleep (300