May
26
By kellabyte  //  3 Comments

Trick to reduce allocations improves response latency in Haywire

On the weekend I was taught a neat, very simple performance optimization you can do in C which I applied to Haywire. I’ve been reducing a lot [...]

Apr
22
By kellabyte  //  1 Comment

Improving the protocol parsing performance in Redis

I’m always looking for side projects that challenge me to become a better developer. Lately these have been C or C++ performance type challenges like Haywire. Last [...]

Mar
12
By kellabyte  //  No Comments

Mencius and Fast Mencius a high performance replicated state machine for WANs

In the ZooKeeper IRC channel someone pointed out to me that one of the authors of ZAB [1] published a paper about a new variant of the [...]

Mar
11
By kellabyte  //  No Comments

Tuning Paxos for high-throughput with batching and pipelining

This is a quick post to link people to a paper I found useful awhile back that discusses the affects of batching and pipelining in Paxos performance [...]

Mar
5
By kellabyte  //  No Comments

Scalable Eventually Consistent Counters

This post is going to be short. I get a lot of requests to share what I’ve learned or implemented but I don’t always have time to [...]

Feb
12
By kellabyte  //  2 Comments

Create benchmarks and results that have value

Different types of benchmarks exist whether its micro benchmarks or other kinds of benchmarks. Benchmark setups themselves are a simulated environment so it’s hard to build a [...]

Feb
9
By kellabyte  //  1 Comment

Routing aware master elections

During a transaction discussion with Selena Marie and Howard Chu on Twitter, Howard mentioned routing protocols which reminded me of an idea I had several months ago [...]

Jan
9
By kellabyte  //  15 Comments

My new test lab

I recently started building my own test lab so that I can run much more reproducible tests and have dedicated hardware that has sufficient capacity to do [...]

Nov
29
By kellabyte  //  6 Comments

Responsible benchmarking

As engineers, I think we have a responsibility with how we publish benchmarks. Not everyone is as much of an expert as you on the things you [...]

Nov
11
By kellabyte  //  1 Comment

Understanding hardware still matters in the cloud

Cloud services and infrastructure from Amazon, Windows Azure and other cloud providers offer resources with high levels of abstraction that sometimes provide scalability and fault tolerance benefits. [...]

Nov
4
By kellabyte  //  3 Comments

The “network partitions are rare” fallacy

I often hear that network partitions are rare in clusters that are up to a couple hundred nodes in size and that you should expect Gossip protocols [...]

Oct
25
By kellabyte  //  5 Comments

Messaging and event sourcing

In the event sourcing communities sometimes I hear anti-messaging statements that messaging increases complexity and messaging increases coupling so you should avoid it. I find this strange [...]

Aug
22
By kellabyte  //  3 Comments

Further reducing memory allocations and use of string functions in Haywire

I wrote previously about how I learned that calling several million string functions like strlen() and strcat() per second really impacted performance in Haywire. Those steps really [...]

Aug
20
By kellabyte  //  4 Comments

HTTP response caching in Haywire

Now that I’ve introduced Haywire I’m going to dig into some of the things I’ve done up to this point to get the performance I’ve gotten. At [...]

Aug
19
By kellabyte  //  4 Comments

Atomic sector writes and misdirected writes

I discovered a very good blog post by J.R. Tipton a Principal Development Lead on ReFS (Resilient File System) at Microsoft. He writes about atomic sector writes and [...]

Aug
19
By kellabyte  //  2 Comments

How memory mapped files, filesystems and cloud storage works

Some databases use memory mapped files inside the database engine to reduce the need to manage an on-disk storage model and in-memory cache model by letting the [...]

Aug
16
By kellabyte  //  4 Comments

Hello haywire

Hello haywire Back in April I decided I was going to learn how to write high performance code and that I was going to go all in [...]

Jul
11
By kellabyte  //  3 Comments

Active Anti-Entropy

Distributed systems that have relaxed consistency guarantees between replicas need ways to eventually converge on the same state. Doing this with a large set of data can [...]

Jul
9
By kellabyte  //  No Comments

Lightning Memory-Mapped Database

A discussion on Twitter broke out about LMDB (Lightning Memory-Mapped Database) and I realized I’ve been using it for quite some time now and haven’t written about [...]

Jul
1
By kellabyte  //  2 Comments

Write amplification

Write amplification or “write cost” is a term used to describe how much physical disk is written to for a logical piece of data in the database. [...]

Jun
7
By kellabyte  //  8 Comments

Amortizing de-duplication at read time instead of write time

Yesterday I was reading a blog post by Jimmy Bogard where he describes using ACID 2.0 (Associative, Commutative, Idempotent, Distributed) to create a ledger for a point of sales system. I’m [...]

Jun
6
By kellabyte  //  4 Comments

LevelDB was designed for mobile devices

In the industry there is a lot of reuse of storage engines because writing a good, reliable and efficient one is a tough challenge and takes a [...]

May
23
By kellabyte  //  No Comments

AMQP and wire format interopability

Today support for AMQP in Windows Azure Service Bus was announced. For more details check out the discussion between Clemens Vasters and David Ingham. Previously I wrote [...]

May
20
By kellabyte  //  2 Comments

Convergent Replicated Data Types

Chris Patterson mentioned CRDT’s on Twitter this morning which reminded me that I haven’t blogged or provided any resource links on them in the past even though [...]