Hi there 馃憢

Welcome to my blog. This is xgugeng, a software engineer for Azure Storage 鈽侊笍鈽侊笍鈽侊笍

Redis Internals: File Event Handling

Events are the core of Redis server, reponsible for dealing with two important tasks: File events: receive the requests from multiple clients, process and responde to clients. Time events: server cron job. In this post, we will take a look at multiplexing event handling processing requests from clients. Clients connect to Redis server via sockets, but the server interacts with these clients only when the sockets can be read or written without blocking. ...

Apr 28, 2019 路 7 min 路 xgugeng

Connection Pool in redis-py

A typical example of using Collection Pools in redis-py is: pool = redis.ConnectionPool(host='localhost', port=6379, db=0) r = redis.Redis(connection_pool=pool) ...

Jan 18, 2019 路 3 min 路 xgugeng

Redis Internals: Dict

Dictionary is a data structure for storing key-value pair. Each key in a dictionary is unique and has a corresponding value. Redis implements its own dictionary and use it for hash key. As a data structure in server side, dictionary must take memory into consideration. That鈥檚 the reason why resizing and rehashing are so significant. ...

Nov 15, 2018 路 9 min 路 xgugeng

Redis Internals: List

Redis provides a generic doubly linked list implementation, which is widely used in Redis. If a list key involves two many elements or there are many long strings in the list, Redis will apply linked list for it. Publish-subscribe, slow query, monitor also make use of linked list. Redis keeps the states of clients in linked list, and client鈥檚 output buffer use it too. This post will plough into the implementation of doubly linked list. ...

Nov 6, 2018 路 6 min 路 xgugeng

Weak References

In Python, objects are never explicitly destroyed. However, they may be garbage-collected when they become unreachable. The del statement deletes names, not objects. An object may be garbage collected as result of a del command, but only if the variable deleted holds the last reference to the object, or if the object becomes unreachable. In CPython, the primary algorithm for garbage collection is reference counting. Essentially each object keeps count fo how many references point to it. As soon as that refcount reaches 0, the object is immediately destroyed: CPython calls the __del__ method on the object (if defined) and the frees the memory. But sometimes, it is useful to have a reference to an object that does not keep it around longer than necessary. Week references to an object do not increase its reference count. A weak reference does not prevent the referent from being garbage collected. ...

Nov 5, 2018 路 2 min 路 xgugeng

Variables Are Labels, Not Boxes

Variables are labels, not boxes. Take the following code for example: a = [1, 2, 3] b = a a.append(4) print(b) If you imagine that variables are like boxes, you cannot make sense of assignment in Python. For an assignment, you must always read the right-hand side first: that鈥檚 where the object is created or retrieved. After that, the variable on the left is bound to the object, like a label stuck to it. Just forget about the boxes. ...

Nov 1, 2018 路 3 min 路 xgugeng

Redis Internals: SDS

C string is used only for literal constant in Redis. For the situation that the string can be modified, Redis provide a Simple Dynamic String(SDS). The key in Redis is actually a SDS, and if you insert a key-value with SET msg "hello world", a SDS instance will be constructed for the key. In this case, the value is also a SDS instance with the content of 鈥渉ello world鈥. Besides string, SDS is also used for buffer which can serve the AOF module and client鈥檚 input. This post will dive into the implementation of SDS comparing with classic C string. ...

Nov 1, 2018 路 8 min 路 xgugeng

What Is the Concurrent Mode Failure?

This is a question that I was asked during an interview: what does the concurrent mode failure means? It鈥檚 really a good question to check your understanding of Concurrent Mark Sweep (CMS) Collector. This post starts with concurrent mode failure and explains the whole steps of one CMS cycle. In a nutshell, The message 鈥渃oncurrent mode failure鈥 signifies that the concurrent collection of the Old Generation did not finish before the Old Generation became full. ...

Oct 31, 2018 路 4 min 路 xgugeng

When Python Executes Decorators

A key feature of decorators is that they run right after the decorated function is defined. This is usually at import time. ...

Oct 29, 2018 路 2 min 路 xgugeng

Design Patterns with First-class Functions

Design patterns are language-independent, but in the context of languages with first-class functions, it鈥檚 beneficial to rethink some patterns. The general idea is: you can replace instances of some participant class with simple functions, reducing a lot of boilerplate code. In this post, I will refactor Strategy using functions objects. ...

Oct 27, 2018 路 4 min 路 xgugeng