Welcome to my blog. This is xgugeng, a software engineer for Azure Storage ☁️☁️☁️
Redis Internals: File Event Handling
Events are the core of Redis server, reponsible for dealing with two important tasks: File events: receive the requests from multiple clients, process and responde to clients. Time events: server cron job. In this post, we will take a look at multiplexing event handling processing requests from clients. Clients connect to Redis server via sockets, but the server interacts with these clients only when the sockets can be read or written without blocking. ...
Connection Pool in redis-py
A typical example of using Collection Pools in redis-py is: pool = redis.ConnectionPool(host='localhost', port=6379, db=0) r = redis.Redis(connection_pool=pool) ...
Redis Internals: Dict
Dictionary is a data structure for storing key-value pair. Each key in a dictionary is unique and has a corresponding value. Redis implements its own dictionary and use it for hash key. As a data structure in server side, dictionary must take memory into consideration. That’s the reason why resizing and rehashing are so significant. ...
Redis Internals: List
Redis provides a generic doubly linked list implementation, which is widely used in Redis. If a list key involves two many elements or there are many long strings in the list, Redis will apply linked list for it. Publish-subscribe, slow query, monitor also make use of linked list. Redis keeps the states of clients in linked list, and client’s output buffer use it too. This post will plough into the implementation of doubly linked list. ...
Weak References
In Python, objects are never explicitly destroyed. However, they may be garbage-collected when they become unreachable. The del statement deletes names, not objects. An object may be garbage collected as result of a del command, but only if the variable deleted holds the last reference to the object, or if the object becomes unreachable. In CPython, the primary algorithm for garbage collection is reference counting. Essentially each object keeps count fo how many references point to it. As soon as that refcount reaches 0, the object is immediately destroyed: CPython calls the __del__ method on the object (if defined) and the frees the memory. But sometimes, it is useful to have a reference to an object that does not keep it around longer than necessary. Week references to an object do not increase its reference count. A weak reference does not prevent the referent from being garbage collected. ...
Variables Are Labels, Not Boxes
Variables are labels, not boxes. Take the following code for example: a = [1, 2, 3] b = a a.append(4) print(b) If you imagine that variables are like boxes, you cannot make sense of assignment in Python. For an assignment, you must always read the right-hand side first: that’s where the object is created or retrieved. After that, the variable on the left is bound to the object, like a label stuck to it. Just forget about the boxes. ...
Redis Internals: SDS
C string is used only for literal constant in Redis. For the situation that the string can be modified, Redis provide a Simple Dynamic String(SDS). The key in Redis is actually a SDS, and if you insert a key-value with SET msg "hello world", a SDS instance will be constructed for the key. In this case, the value is also a SDS instance with the content of “hello world”. Besides string, SDS is also used for buffer which can serve the AOF module and client’s input. This post will dive into the implementation of SDS comparing with classic C string. ...
What Is the Concurrent Mode Failure?
This is a question that I was asked during an interview: what does the concurrent mode failure means? It’s really a good question to check your understanding of Concurrent Mark Sweep (CMS) Collector. This post starts with concurrent mode failure and explains the whole steps of one CMS cycle. In a nutshell, The message “concurrent mode failure” signifies that the concurrent collection of the Old Generation did not finish before the Old Generation became full. ...
When Python Executes Decorators
A key feature of decorators is that they run right after the decorated function is defined. This is usually at import time. ...
Design Patterns with First-class Functions
Design patterns are language-independent, but in the context of languages with first-class functions, it’s beneficial to rethink some patterns. The general idea is: you can replace instances of some participant class with simple functions, reducing a lot of boilerplate code. In this post, I will refactor Strategy using functions objects. ...