A quick post on adapting your approach to the problem at hand, not just following the same ideas indiscriminately.

We Are .NET

Processing hundreds of thousands of items one-by-one with individual database queries is a common performance anti-pattern. A real-world batch job that took nearly 24 hours was reduced to ~15 minutes by bulk-loading data grouped by day instead of querying per item. The post generalizes this to other I/O-heavy scenarios (HTTP APIs, messaging, file I/O), warns against chatty I/O patterns like ORM lazy loading in web requests, and discusses why parallelization or moving logic into the database are less effective than batching. It also cautions against over-estimating scale and stresses understanding your tools, using Entity Framework Core's change tracker and lazy loading proxies as an example of hidden memory overhead.

Sometimes it's just better to load "all" the data

<p>We’ve found for several applications fetching all the data and just caching it in memory or redis to be much more performant that the constant chatter to whatever system it is. This is especially important when dealing with a 3rd party api or something</p>
