/blogs/Recent content in Blog on Abhinand JhaHugoen-usFri, 12 May 2023 00:00:00 +0000/blogs/2023-05-12-borg/Fri, 12 May 2023 00:00:00 +0000/blogs/2023-05-12-borg/Reference papers: [1] Large-scale cluster management at Google with Borg Summary In this paper, the authors describe a cluster-management system called Borg, that is responsible for scheduling tens of thousands of jobs running on Google’s warehouse scale data centers. The authors elaborate on the key design principles behind Borg and highlight how it has improved the efficiency and reliability of Google’s data centers. Some of the challenges faced in designing such a system are discussed by the authors such as large heterogeneous workloads, optimization of resource utilization, the need for high availability and low-latency./blogs/2023-04-12-xen-vm/Wed, 12 Apr 2023 00:00:00 +0000/blogs/2023-04-12-xen-vm/Reference papers: [1] Xen and the Art of Virtualization Summary Barham et al. introduce a Virtual Machine Monitor (VMM) called Xen. The main contribution of their approach is their implementation of paravirtualization – modifying the guest OSes to provide efficient virtualization efficiency. The authors first introduce the concept and need for virtualization and the various existing approaches by which virtualization is achieved. The authors then give an overview of the Xen architecture and how it virtualizes memory, CPU and IO./blogs/2022-05-12-paxos/Thu, 12 May 2022 00:00:00 +0000/blogs/2022-05-12-paxos/Reference papers: [1\ PAXOS Made Live: An Engineering Perspective] Summary Paxos is a well known and researched consensus protocol to obtain fault tolerance in distributed systems. In this paper, Chandra et al. highlight their experience in implementing the Paxos algorithm to achieve fault-tolerant replication in distributed systems in a production environment. They show that even though the algorithm is well researched in academic literature, its practical implementation is still a quite involved process./blogs/2022-03-10-live-migration/Thu, 10 Mar 2022 00:00:00 +0000/blogs/2022-03-10-live-migration/Reference papers: [1] Live Migration of Virtual Machines Summary Clark et al. propose an approach to migrating live VMs across different instances with minimal degradation of the quality of service and downtime. Live migration is particularly helpful for load re-balancing, fault management and server maintenance. The authors first describe the traditional approaches to live migration including stop-and-copy, demand- copy, pre-copy and other hybrid methods. The authors adopt pre-copy as their approach because of its efficiency./blogs/2022-02-10-gfs/Thu, 10 Feb 2022 00:00:00 +0000/blogs/2022-02-10-gfs/Reference papers: [1] The Google File System [2] GFS: Evolution on Fast-Forward Summary One of the main challenges in the domain of distributed systems is building scalable storage systems that can handle large amounts of data in an efficient way. The Google File System (GFS) proposed in [1] is a scalable distributed file system for large scale data-intensive applications. Ghemawat et al. highlight some of the key design and performance considerations that went into developing GFS./blogs/2022-02-05-chord/Sat, 05 Feb 2022 00:00:00 +0000/blogs/2022-02-05-chord/Reference paper: Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Summary One of the fundamental problems in peer-to-peer (P2P) applications is to efficiently locate a particular node in the network. Stoica et al. present the design for a scalable and efficient P2P distributed lookup protocol named Chord. The authors first give an overview of consistent hashing which is widely used in such scenarios. They highlight the scalability issues of consistent hashing that are caused by each node having to maintain the state of all of the other nodes in the network./blogs/2022-01-24-tail-at-scale/Mon, 24 Jan 2022 00:00:00 +0000/blogs/2022-01-24-tail-at-scale/Reference paper: The Tail at Scale Summary One of the challenges in building large-scale systems is to consistently maintain low-latency in responses. Dean and Barroso, in their paper highlight techniques that can be used to build distributed systems that have an overall low-latency despite its constituent components having occasional high-latency. The authors first provide several reasons that contribute to high tail latency such as shared resources, queuing, daemons, garbage collection etc./blogs/2022-01-20-server-design-advanced/Thu, 20 Jan 2022 00:00:00 +0000/blogs/2022-01-20-server-design-advanced/Reference papers: [1] A Scalable and Explicit Event Delivery Mechanism for UNIX [2] accept()able Strategies for Improving Web Server Performance Summary Server performance depends on a number of factors and in order to design high performance and scalable servers, it is important to carefully reason about these factors. Authors of [1] and [2] argue that a server’s policies for accepting new client connections [2] and the OS event-notification mechanism [1], have a signif- icant effect on a server’s performance./blogs/2022-01-16-server-design-basic/Sun, 16 Jan 2022 00:00:00 +0000/blogs/2022-01-16-server-design-basic/The reference paper can be found here. Summary In this paper, the authors propose Flash – an efficient and portable web server that utilizes an asymmetric multi-process even-driven (AMPED) architecture. This architecture is a combination of the single-process event-driven (SPED) and multi-process/multi-thread (MP/MT) architectures. Flash consists of a main process that handles HTTP requests in an event-driven fashion, and it uses several asynchronous helper processes to perform blocking operations on the disk./blogs/2022-01-14-end-to-end/Fri, 14 Jan 2022 00:00:00 +0000/blogs/2022-01-14-end-to-end/The reference paper can be found here. Summary In this paper, the authors provide an argument that follows the “Occam’s razor” principle in the context of designing systems. The central problem that the authors address is deciding the appropriate boundaries between the functions of a distributed system, and where to place these functions in the system. The end- to-end argument aims to make the lower levels of a system as simple as possible by implementing functions closer to the endpoints.