Opened 7 years ago

Last modified 9 months ago

#95 assigned defect

Memory is not free()'d after xcache_get() data is unset

Reported by: krogot Owned by: moo
Priority: major Milestone: 3.2.0
Component: cacher Version: 1.2.2
Keywords: Cc: mat999@…
Application: PHP Version: 5.3.2
Other Exts: SAPI: Irrelevant
Probability: Always Blocked By:
Blocking:

Description (last modified by moo)

I've tested EA vs XCache performance on cache reads and got the following problem:

Code:

<?php
    function ea_read ($max) # void
    {
        global $DATA;

        for($i = 0;$i<10;$i++)
        {
            $key[$i] = 'some_key_'.$i;
            eaccelerator_put($key[$i], $DATA);
        }

        for($i = 0;$i<$max;$i++)
            $val = eaccelerator_get($key[$i%10]);
    }

    function xcache_read ($max) # void
    {
        global $DATA;

        for($i = 0;$i<10;$i++)
        {
            $key[$i] = 'some_key_'.$i;
            xcache_set($key[$i], $DATA);
        }

        for($i = 0;$i<$max;$i++)
        {
            $val = xcache_get($key[$i%10]);
        }
    }

Result ($max = 10000):

<?php
    $DATA = str_repat('x',10000); 

Everything is ok.

<?php
    $DATA = array_map('uniqid', range(1,10));

Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 15 bytes).

It seems that XCache somehow marks array items that they aren't released (may be it increases ref_count?). I tried to add "unset($val);" after "$val = xcache_get();" but it didn't help.

The only solution I've found is:

<?php
   xcache_set($key,serialize($data));
   $val = unserialize(xcache_get($key));

but it looks weird (and slows down xcache calls) ...

Change History (12)

comment:1 Changed 7 years ago by moo

  • Description modified (diff)
  • Status changed from new to assigned

comment:2 Changed 7 years ago by moo

i have a error in theory about this issue. it will take some time for me to find a correct way

comment:3 Changed 7 years ago by moo

  • Milestone set to 1.3.0
  • pending set to 0
  • SAPI set to Irrelevant

comment:4 Changed 7 years ago by moo

  • Summary changed from Memory is not free()'d to Memory is not free()'d after xcache_get() data is unset

comment:5 Changed 7 years ago by oli

Hi,

I am not quite sure, but I think this was the problem I was facing earlier today. It could be, that report:9 is also the same. Consider the following code:

<?php
$data='Some Content';
//xdebug_debug_zval('data');
xcache_set('example', $data, 3600);

$data=xcache_get('example');
//xdebug_debug_zval('data');

for($i=0;$i<=1000000;$i++){
	$var=xcache_get('example');
	if(($i%10000)==0) echo "Memory consumption after $i iterations is " . memory_get_usage() . "\n";
}
?>

Running this under two distinct systems both having PHP 5.2.5 together with XCache 1.2.2 results in

PHP Fatal error:  Allowed memory size of 33554432 bytes exhausted (tried to allocate 16 bytes) in ... on line 10

after ~700000 iterations. Well, it was assumed that xcache_get holds a reference which makes it impossible to free the variable ($var). I tried to verify this by using XDebug on one of the two machines. If you enable the two xdebug_debug_zval statments you'll get something like

data: (refcount=1, is_ref=0)='Some Content'
data: (refcount=2, is_ref=0)='Some Content'

which is kind of surprising. It seams the reference counter of the variable retrieved from xcache equals one plus the reference counter when setting the variable (originally I had a variable having refcount=3 and after retrieving it from xcache it had refcount=4). Replacing the variable in xcache_set through a constant string gives you a variable with refcount=1, which is actually correct (no memory leak). I don't know much about the internals of PHP, but it seams understandable that the memory is not freed since the variable reference counter too big.

I also find a (temporary) solution for that problem. Replacing the xcache_set statement through

xcache_set('example', (string)$data, 3600);

seams to fix it. I assume the explicit cast to string lets PHP create a temporary variable with refcount=0 (similar to a constant).

Please contact me, if you need any further information.

comment:6 Changed 7 years ago by moo

thanks. the problem is confirmed already. i was trying to figure out a fix

comment:7 Changed 7 years ago by oli

Thank you for looking at the problem. I was aware this was already confirmed. However, I hope the additional input makes it easier to locate the issue, and if not at least affected user can use the explicit cast trick when setting their variables as long as the bug is not fixed.

comment:8 Changed 7 years ago by moo

hrm... you're right. the workarround can be done inside xcache_set. i'll check if the workarround fix the problem correctly. referenced variable is the key point.

comment:9 Changed 7 years ago by moo

sorry, it does not work

<?php
$b = "a";
$a = array(&$b, &$b);
$c = (array) $a;
$c[0] = 1;
var_dump($b);
?>

actual result: int(1), expected: string(1) "a"

and btw, the problem is located already.

i just want to figure out a most optimized way to do it by paying more time thinking about it. this problem is not urgency because no one want to do dead or huge loop in webserver and the leak will be free'ed by php GC anyway.

comment:10 Changed 4 years ago by splitice

  • Cc mat999@… added
  • PHP Version changed from 5.1.6 to 5.3.2
  • Probability set to Always
  • Version changed from 1.2-dev to 1.2.2

Finally, who would guess I had to activate my email to post here.

Developed my own fix, not the best way to fix it, really I should be fixing it in xcache_set, howeaver my attempts at copying the zval and changing refcount where not successfull and as such I have resorted to doing so in xcache_get

Patch outlined: http://thewarezscene.org/forums/the-warez-scene-releases-f219/xcache-xcache-get-memory-leak-fix-t645163.html

FIND:
if (!VAR_ENTRY_EXPIRED(stored_xce)) {

found = 1;

AFTER, ADD:
Z_SET_REFCOUNT_P(stored_xce->data.var->value,1);

comment:11 Changed 16 months ago by moo

  • Milestone changed from 3.0.2 to 3.1.0

comment:12 Changed 9 months ago by ablagoev

With the introduction of cli support for xcache the problem becomes more relevant. If you are running daemons which use xcache_get you should watch out for this. A simple gc_collect_cycles() resolves the issue.

More info here:
http://www.php.net/manual/en/function.gc-collect-cycles.php

Note: See TracTickets for help on using tickets.