ASP.NET Core–Read data from the HttpClient in a memory efficient way

Baca Juga

• ASP.NET Core–Read data from the HttpClient in a memory efficient way
• REST design pattern
• .NET Conf 2021 - Sessions, slides and demos are online
• .NET Core–The case of the disappearing authorization header
• Keep your project dependencies up to date with dotnet outdated
• Front-End Performance Checklist 2021

I created a service that allowed users to download all address data from Belgium in CSV format. By default when fetching this data through the HttpClient it would result in having the full blob into memory.

But thanks to the power if IAsyncEnumerable<> and setting the HttpCompletionOption.ResponseHeadersRead flag when calling GetAsync() I could get this done in a very efficient way:

	public async IAsyncEnumerable<Address> ReadLines(IHttpClientFactory httpClientFactory, string uri)
	{
	var httpClient = httpClientFactory.CreateClient();
	using var response = await httpClient.GetAsync(
	uri, HttpCompletionOption.ResponseHeadersRead);
	response.EnsureSuccessStatusCode();
	await using var stream = await response.Content.ReadAsStreamAsync();
	using var streamReader = new StreamReader(stream, Encoding.UTF8);
	while (!streamReader.EndOfStream)
	{
	var line = await streamReader.ReadLineAsync();
	var address = GetAddress(line);
	yield return address;
	}
	}

view raw ReadAddresses.cs hosted with ❤ by GitHub

By setting the HttpCompletionOption.ResponseHeadersRead flag the client would only wait until the headers are received and then continue execution. So I could start processing the results before all data was downloaded.