Tiwariji writes.

I write to understand. Mostly computers, philosophy, music and systems.
Writings

i want better conversations

when i was a kid i didn’t talk much. i have NO idea why. this is something that i think about very often. when you just spawned right, when you’re like one minute old, what information do you have? what do you have in you that makes you different from others? aren’t all those kids the same? the little ones. minute old ones. i just LLM’ed this and i feel dumb. obviously all the kids are different even before they are born. but anyways, that’s not why im writing this(disclaimer: i dont know why im writing this) ...

Big integer in Python

Today we are looking at how you can work with big integers in Python. What do I mean by “big” integers? A standard 32-bit integer tops out at 2,147,483,647. A standard 64-bit integer tops out at about 9.2 × 10^18. The very interesting thing about Python is: it doesn’t have a fixed ceiling for integers! The only limitation is the machine’s memory. So you can effectively store any integer in a Python int. ...

desire to be significant

I (we) have a burning desire to be significant as a human. Maybe not everyone has it, I do. By time I’ve realized this pleasure is pretty hard to achieve. The bar is too high. can you be happy if you’re significant? we know a lot of people who were so significant to society but not happy. Tesla struggled with business acumen and was often outmaneuvered by profit-driven contemporaries. He spent his final decades living in a series of New York hotels, steadily running out of money while working on increasingly eccentric, unfunded projects. He died practically penniless and alone in his room at the New Yorker Hotel in 1943. Van Gogh is one of the most famous and influential figures in the history of Western art. His expressive use of color and bold brushwork profoundly impacted expressionism and the trajectory of modern art. His life was filled with severe poverty, malnutrition, and worsening mental health crises. He sold only one painting during his lifetime and spent his final years considering himself an absolute failure. He died by suicide at the age of 37 in 1890, entirely unaware of the immense legacy he would leave behind. there’s one more: The theoretical father of computer science and artificial intelligence, Turing’s work in cracking the Enigma code during World War II arguably shortened the conflict by years and saved millions of lives. His concept of the “Universal Turing Machine” laid the foundation for all modern computing. In 1952, Turing was prosecuted by the British government for homosexual acts. To avoid prison, he accepted chemical castration, which severely impacted his physical and mental health. Stripped of his security clearance and treated as a criminal by the country he helped save, Turing died in 1954 from cyanide poisoning. An inquest ruled it a suicide. All of these “significant” people died, a catastrophic death. They didn’t know what they “left” behind. ...

I use arch btw

Story Time I have been using ubuntu for the past 3 years or so, and honestly I love it. When I initially installed ubuntu it was out of necessity. I had an old laptop and I wanted to do machine learning, which at that time required a GPU. Today you can just use cloud GPUs and pretty decent prices at that! Since ubuntu is the most light operating system and beginner friendly, I thought that’d be a decent choice. ...

LSTM from scratch in PyTorch

Today we’re looking at LSTM (Long Short Term Memory) neural networks. The standard for sequential data was RNNs (Recurrent Neural Networks). RNNs had an issue. They were good at remembering things, but they… well, they kept forgetting too! They had what we call a vanishing gradient problem. Small example to guide your visual senses: The spine of the problem is, when the gradients flow backwards, they get multiplied. And say if that number that we’re multiplying with is smaller - which a lot of times the gradients are, then at some point the gradient will be close to zero and boom you’ve lost signal. At each timestep, the gradient gets multiplied by the derivative of the activation function (like sigmoid) — a number that maxes out at 0.25. Ten steps back, your gradient is 0.25¹⁰ — essentially zero. ...

Implementing mini Ollama from scratch

In my previous work, we discussed different ideas and ways in writing a server. To quickly recap, it’s writing it asychronously, writing a multi-threaded server or writing a “dumb” one. Each one has its pros and cons. People might say writing an async server would solve issues but it would not be the complete story. An async server helps when threads are idle waiting on I/O — network reads, file reads from external devices. Things like this, you can build an async server and it’ll get you the fastest latency. But they don’t provide us with the same improvements when the work is not I/O bound. Forge’s threads are idle waiting on compute — a different problem. We’ll see why in this article. ...

Writing an inference server in bare-metal C++

What is a Server? It is a software program that manages resources over a network. The whole network could be visualized like this: Today we are writing a server. A dumb one, at first. We’ll benchmark the load handling of the server. And then we optimize. Difference between websockets and a http server model WebSockets maintain a persistent, stateful connection where both client and server can continuously exchange data. In a traditional HTTP model, the client usually sends a request, receives a response, and the connection is then closed. ...

Studying threads and benchmarking python vs c++

Threads are the smallest unit of execution that an operating system can schedule inside a process. People (me) get confused about what threads are and what processes are. This article will talk about threading as a programming concept and not the theory behind processes and threads. Nevertheless, we’ll talk about processes and threads too. Processes and threads Process A ├── Thread 1 ├── Thread 2 └── Thread 3 Process B ├── Thread 1 └── Thread 2 Process is an independent program instance. It has its own memory and resources. Think of it like an agent. A process can have one or more threads in it. Threads share the process memory and resources, though each thread has its own execution state and stack. How processes and threads are implemented depends on specific programming languages and operating systems, so I highly recommend checking out the wiki. ...

From scratch Quantization in neural networks

“compute solves a lot of problem” If we just had enough compute, a lot of the problems that we experience today would be solved. Loger contexts, smarter weights and biases, etc. But right now we don’t have infinite compute. That’s the sad reality. So we optimize. Quantization is our attempt at just that. history Reference: https://arxiv.org/abs/2103.13630 Quantization is a way of compression. It is a process of mapping a large set of continous or high-precision values into smaller discrete set of values. ...

How PolyBlocks lowers tensor programs for GPUs

Recently, at PyTorch Day India in Bangalore, I saw a talk on AI compilers. Here is the link: YouTube Picture from the session I didn't know there were Indian labs working on the AI compiler problem. But it turns out there are. PolyMage Labs is an IISc lab in Bangalore working on PolyBlocks. Since AI is moving fast, there is a clear need for efficient AI compilers that can lower high-level tensor programs to IR for GPUs, TPUs, and other backends. PolyBlocks minimizes dependency on external vendor libraries like cuBLAS/cuDNN while still generating highly optimized code via compiler-driven transformations and tiling. ...

Tiwariji writes.#

Tiwariji writes.