comp.lang.ada
 help / color / mirror / Atom feed
From: "Luke A. Guest" <laguest@archeia.com>
Subject: Re: How to read in a (long) UTF-8 file, incrementally?
Date: Wed, 3 Nov 2021 08:48:58 +0000	[thread overview]
Message-ID: <sltigk$43o$1@gioia.aioe.org> (raw)
In-Reply-To: d1c5ba75-bc0a-4e7b-a2df-394bc710cbcen@googlegroups.com

On 02/11/2021 17:42, Marius Amado-Alves wrote:
> As I understand it, to work with Unicode text inside the program it is better to use the Wide_Wide (UTF-32) variants of everything.

You can take a look at my simple lib: https://github.com/Lucretia/uca

> Now, Unicode files usually are in UTF-8.
> 
> One solution is to read the entire file in one gulp to a String, then convert to Wide_Wide. This solution is not memory efficient, and it may not be possible in some tasks e.g. real time processing of lines of text.

It can read into a large string buffer.

> If the files has lines, I guess we can also work line by line (Text_IO). But the text may not have lines. Can be a long XML object, for example.

And can break it up into lines. There's no Unicode consistency checks.

The lib is a bit hacky, but seems to work for now. There's nothing more 
than what I've mentioned so far.

  parent reply	other threads:[~2021-11-03  8:48 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-02 17:42 How to read in a (long) UTF-8 file, incrementally? Marius Amado-Alves
2021-11-02 18:17 ` Dmitry A. Kazakov
2021-11-03  7:43 ` Vadim Godunko
2021-11-03  8:48 ` Luke A. Guest [this message]
2021-11-04 11:43   ` Marius Amado-Alves
2021-11-04 12:13     ` Dmitry A. Kazakov
2021-11-04 14:30     ` Luke A. Guest
2021-11-05 10:56       ` Marius Amado-Alves
2021-11-05 19:55         ` Simon Wright
2021-11-16 11:55           ` Marius Amado-Alves
2021-11-16 12:36             ` Dmitry A. Kazakov
2021-11-16 13:52               ` Marius Amado-Alves
2021-11-16 20:23               ` Randy Brukardt
2021-11-16 15:25             ` Luke A. Guest
2021-11-16 17:38             ` Vadim Godunko
replies disabled

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox