Opened 8 months ago

Closed 8 months ago

Last modified 7 months ago

#333 closed defect (fixed)

Memory usage keeps growing while repeatedly require() the same empty file

Reported by: xubin04 Owned by: moo
Priority: major Milestone: 3.2.0
Component: cacher Version: 3.0.4
Keywords: memory Cc:
Application: PHP Version: 5.4.21
Other Exts: SAPI: Irrelevant
Probability: Always Blocked By:
Blocking:

Description

Hi, mOo, thanks for your great work and insistency during the past several years on Xcache, We've been using Xcache since its early versions, after upgrading Xcache from 1.3.2 to 3.0.4 recently, we met with a possible memory management issue in Xcache.

Code:

a.php

<?php
      for ($i=0; $i<10000; $i++) {
            require('b.php');
            printf('%d\n",  memory_get_usage());
      }

b.php

<?php
// this could be an empty file

Then, we run a.php with the following command:

php -d xcache.test=on a.test

(I set xcache.test=on since Xcache doesn't enable cache under CLI mode by default)

and then the memory usage keeps growing:

224184
224512
224808
225104
225400
225696
225992
226288
226584
226880
227176
227472
227768
228064
...............

I agree it's far from PHP's best practices to include a file repeatedly, however, can we avoid extra memory allocation in such circumstances? The allocation seems to be reside in xc_cacher.c:

2147     /* found entry */
2148     if (stored_entry && stored_php) {
2149         zend_llist_add_element(&CG(open_files), h);
2150         return xc_compile_restore(stored_entry, stored_php TSRMLS_CC);
2151     }

Is this a bug or something 'have to be so' ? Is there an approaches we can bypass this issue if the application code is beyond our control?

Regards!

This is my PHP version info:

PHP 5.4.21 (cli) (built: Dec  6 2013 19:30:14)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies
    with XCache v3.0.4, Copyright (c) 2005-2013, by mOo
    with XCache Cacher v3.0.4, Copyright (c) 2005-2013, by mOo

Change History (9)

comment:1 Changed 8 months ago by xubin04

  • Summary changed from Memory keeps growing while repeatedly require() the same file to Memory usage keeps growing while repeatedly require() the same empty file

comment:2 Changed 8 months ago by moo

can you reproduce not not without XCache?

comment:3 Changed 8 months ago by moo

  • Milestone changed from undecided to 3.2.0
  • Status changed from new to accepted

comment:4 Changed 8 months ago by moo

  • Resolution set to fixed
  • Status changed from accepted to closed

In 1478:

fixed #333: reduce memory usage for small or empty files

comment:5 Changed 8 months ago by moo

In 1480:

MFT, fixed #333: reduce memory usage for small or empty files

comment:6 Changed 8 months ago by xubin04

Really impressed by your efficiency.
This issues doesn't trouble much except for scripts under CLI mode.
Unfortunately we have a whole bunch of PHP scripts running under CLI.
In order to use xcache_isset() / get() / set() under CLI mode , we've always been keeping xcache.test=on, then many scripts quits because memory used out.
I've noticed your former posts on the xcache.test option, and I did some experiments:

  1. Under 3.0.4, I can turn xcache.test=off to avoid this memory allocation issue, but calling xcache functions, suck as xcache_isset() will leads to warning.
  2. With 3.1.0, I can turn xcache.test=off while still keep xcache_isset()/ xcache_get()/ xcache_set() callable.

If we want to :

  1. Avoid unnecessary memory growth for CLI scripts
  2. Avoid warnings message from calling xcache_isset().... in CLI mode

Is:

  1. Choose Xcache 3.1.0
  2. turn xcache.test=off

the best approach?

comment:7 Changed 7 months ago by moo

Sorry to be so late late catching this topic up. I'm not sure what problem remains there after the fix. The fix is not applied on 3.0.4 (to release 3.0.5, if any) nor applied on 3.1.0 yet because the leak is not huge for server usage and can be recycle by zend memory manager. I'd let it leek instead of taking the chance to break 3.0.x 3.1.x

I haven't look into 3.1.0 why xcache.test=off make no leak about the bug reported in this ticket as seems your reply implies. Your best bet would be back port the changes to 3.0.x or 3.1.x as you like if you don't take trunk or 3.2.x, and build your own XCache binary

Feel free to reply if you're misunderstood.

comment:8 Changed 7 months ago by xubin04

Hi mOo
Thanks for your reply.You reply is clear enough, it was my last comment which confused you.
What I would like to say in my last comment is about PHP scripts running in CLI mode.
With XCache 3.1.0, we can set xcache.test=off to disable opcode cache under CLI mode, and thus bypass the memory usage problem. While we can’t do this under 3.0.4, since our framework uses xcache_isset(), and it will leads to WARNING if XCache.test=off.
So I’m really glad to see this small change on xcache_isset() in XCache 3.1.0.

comment:9 Changed 7 months ago by moo

Thanks for your reminding me about xcache.test.

yes, you can and should always have xcache.cacher = off, and/or xcache.size=0 to disable opcode cacher (although cli + xcache.test=off will too disable opcode cacher for you already, because many people tends to misunderstand cgi/cli/fastcgi)

and yes, with xcache.test=off, var cache is also disabled for cli before 3.1.0. which is realized wrong, so was corrected in 3.1.0 to be not affected by xcache.test (reads as: xcache.test is no longer needed for you). you can turn var cache on or off by using xcache.var_size.

The answer to "Is: Choose Xcache 3.1.0 + xcache.test=off the best approach?" would be "use 3.1.0 + xcache.cacher = off, leave xcache.test=off by default"

And again if you insist on using 3.0 not 3.1, you can back port the patch from 3.1 :)

Note: See TracTickets for help on using tickets.