How to determine Redis memory leak? -

How to determine Redis memory leak? -

our redis servers are, since yesterday, gradually (200mb/hour) using more memory, while amount of keys (330k) , info (132mb redis-rdb-tools) remain same.

output of redis-cli info shows 6.89g used memory?!

redis_version:2.4.10 redis_git_sha1:00000000 redis_git_dirty:0 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.6 process_id:3437 uptime_in_seconds:296453 uptime_in_days:3 lru_clock:1905188 used_cpu_sys:8605.03 used_cpu_user:1480.46 used_cpu_sys_children:1035.93 used_cpu_user_children:3504.93 connected_clients:404 connected_slaves:0 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 used_memory:7400076728 used_memory_human:6.89g used_memory_rss:7186984960 used_memory_peak:7427443856 used_memory_peak_human:6.92g mem_fragmentation_ratio:0.97 mem_allocator:jemalloc-2.2.5 loading:0 aof_enabled:0 changes_since_last_save:1672 bgsave_in_progress:0 last_save_time:1403172198 bgrewriteaof_in_progress:0 total_connections_received:3616 total_commands_processed:127741023 expired_keys:0 evicted_keys:0 keyspace_hits:18817574 keyspace_misses:8285349 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:1619791 vm_enabled:0 role:slave master_host:***blocked*** master_port:6379 master_link_status:up master_last_io_seconds_ago:0 master_sync_in_progress:0 db0:keys=372995,expires=372995 db6:keys=68399,expires=68399

the problem started when updated our (.net) client code booksleeve 1.1.0.4 servicestack v3.9.71 prepare upgrade redis 2.8. lot of other stuff updated , our session state store (also redis, harbour client) not show same symptoms.

where redis memory going? how can troubleshoot it's usage?

edit: restarted instance , memory returned 350m , climbing again. top 10 largest objects still same size, ranging 100k 25m nr 1. amount of keys has dropped 270k (330k earlier).

here sources of "hidden" memory consumption in redis:

marc mentioned buffers maintained master feed slave. if slave lagging behind master (because runs on slower box instance), memory consumed on master.

when long running commands detected, redis logs them in slowlog area, takes memory. may want utilize slowlog len command check number of records have here.

communication buffers can take memory. far remember, old versions of redis (and 2.4 quite old - should upgrade), unbounded, meaning if transfer big object @ point, communication buffer associated client connection grow , never shrink. if there many clients dealing big objects, possible explanation. if utilize commands retrieving big info redis (in 1 shot), can explanation well. instance, simple keys * command applied on redis server storing millions of keys consume important amount of memory.

you mentioned have objects big 25 mb. have 404 client connections, if each of them needs access such objects @ point in time, consume 10 gb of memory.

memory-leaks redis stackexchange.redis