前一阵子在得道 APP 上读完了这本《PyTorch 性能与显存优化手册》，感觉是一本很不错的 PyTorch Training 入门读物，很适合刚刚接触这个领域的新手小白来读；同时整本书也提供了一个 PyTorch Training 的优化大纲，可以作为一个引子来扩展去学习更加底层的知识和技术。 这篇帖子主要是我读的时候标记下来的一些知识点，并没有包含书籍里面的全部内容。强烈推荐大家去阅...

User sequence modeling has been a hot topic recently in recommendation system thanks to the advancement of transformer architecture and more powerful hardware. In this blog, I would like to have a ...

In this post, I would like to provide a simple summary on the papers I have read in the first quarter of 2025 and discuss some of my thoughts on recent trend regarding recommendation system. Here i...

In this post, I would like to discuss how to build slack, a very popular realtime messaging application, especially for group messaging (a.k.a channel) in cooperation messaging scenario. Functiona...

In this post, I would provide a quick summarization to some papers from Recsys 2024 which I think is pretty interesting and worth a read. The list of paper is selected based on my personal research...

In my pervious post, I have briefly mentioned about sparse features and how they could be used in recommendation system. In this post, let’s have a deeper look into sparse features, as well as revi...

I have been a fan of online game since my parent bought me a PC when I was in 5th grade. I could still feel the excitement when every Thursday night I rushed home and played DaHuaXiYou 2 with my fr...

Reading open source code has been a recommended approach for software engineers to learn. However, in my past 8 years career, I didn’t do a good job on that. After working in a startup for 1 year, ...

这个是读完这篇综述 Efficient Training of Large Language Models on Distributed Infrastructures - A Survey 之后的一个产出，这篇综述文章针对 LLM 的 training 介绍的已经很详细了，但是同时内容过多也不可能全都学完。这里针对自己整理的一些笔记来列一个之后学习的提纲，这个提纲肯定是非常主观的，推荐大家...

最近读了一本和 distributed system 相关的书籍，介绍了在 distributed system 里面常用的一些 pattern. 这是一篇简要的读书笔记，把书中提到的几个 pattern 总结了下来; 我计划会经常更新这篇 blog, 把我新学习到的或者总结出来的一些 pattern 记录在这里; 希望能起到一个引导性的作用，给大家提供一个提纲挈领的思路 Patterns...