Reorganization of random package

[denizzzka]

I don't like the way the module std.random of Phobos 2 is designed

From a software side we have three types of ~random numbers sources:

1. Hardware (or environmental) "true" noise.
   The most reliable source, which may not be that fast, i.e. it may be emptied in some cases, so it can be blocking and non-blocking in block devices terms.
   This is exacatly what provided by /dev/random in Linux. It should also be noted that this random source type does not exist on all platforms.

2. Based on hardware noise (described in 1) seeded pseudo-random sequence.
   Less (but still) reliable because it extrapolates a true random number using a deterministic algorithm (like described in 3 below). Suitable for generating large volumes of numbers.
   This is exacatly what provided by /dev/urandom in Linux. Again, not all platforms providing it.

3. Pre-determined pseudorandom sequences based on some predictable algorithm.
   Also good for getting large amounts of numbers and fast, but it strictly can't be used for cryptography etc. Its advantage is that it is always available in all systems since (at worst) it is just a mathematical function.

I think if we save users from deepening into details this will only go to the benefit of security. So, my suggestion:

1. Do not provide any access about entropy sources (I am about std.internal.entropy.EntropySource.tryAll and forceEntropySource()). At the application programming level we usually have one source of true entropy. (If this is not so please correct me.) No need to make ambitious interfaces describing the theoretical diversity of RNGs. The "entropy" word can be excluded from the API description completely.

2. Completely exclude "seeding" concept: this is a source of potential issues (dlang/phobos@3c2f87e). Seeding can be encapsulated inside of of urandom generator (see 2 above) if needed.

In fact, you know exactly what amount and quality of random bytes you want to get at some point of your code. And, for example, if system does not provides true RNG needed by you, then let the corresponding function be totally unavailable for compilation and leads to compile time error. Then you can't accidentally build your neat designed software with weak predictable RNG.

It follows from this that it is necessary to provide only four points for obtaining random numbers, all without the need for any combining of them by users. (My suggestion is place each of it in dedicated std.random.* module)

1. std.random.truerandom: implemented as OS/hardware call if system provides hardware (or environmental) random number generator. Suitable for encryption key generation, etc. Maybe three functions will be provided, something like:

ubyte[Size] trueRandom(size_t Size)(); // blocks and waits if no enough entropy
bool trueRandom(ref ubyte[] result); // returns false if no enough entropy
void trueRandomEx(ref ubyte[] result); // throws if no enough entropy

If there is no random number generator in the system, then these functions will not be available and the compilation may end with the error!

2. std.random.seededrandom: function(s) that is implemented either by OS call (for Linux/Windows/Mac) or by some another TRNG call + PseudoRandom (on baremetal platforms). Does not blocks and not throws. Does not exist if there is no TRNG avalable because internally uses seed value. So, again: if there is no random number generator in the system, then these functions will not be available and the compilation may fail.
   These are pretty good random numbers for general purpose like UUID generation.

3. std.random.pseudorandom: not for cryptography at all. Name was specifically chosen so that the user would clearly see the "pseudo" prefix.
   Suitable for drawing a starry sky in a retro games or so one. Internally calls std.random.seededrandom if exists or uses std.random.predetermined (4) with seed value if seededrandom doesn't exist. Guaranteed to exist on all platforms.

4. std.random.predetermined: functions implementing pseudorandom number generators (PRNG). Mostly for internal use, but sometimes users may want to get guaranteed repeatability of pseudorandom sequences.

That's all, and nothing superfluous! I.e., if you do not use something, it does not creates any global variables, etc. From the point of view of the user, seems, everything is also simple and clear. It will be difficult to make a issue with RNG in this case.

[pbackus]

This post appears to contain misinformation about /dev/random and /dev/urandom. Specifically, the idea that /dev/random is "true" randomness and /dev/urandom is "psuedorandom" is a myth; in reality, they are both based on the output of a CSPRNG.

For more details, see "Myths about /dev/urandom".

[denizzzka]

Fact: /dev/random has a very nasty problem: it blocks.

But that’s good! /dev/random gives out exactly as much randomness as it has entropy in its pool. /dev/urandom will give you insecure random numbers, even though it has long run out of entropy.

Fact: No. Even disregarding issues like availability and subsequent manipulation by users, the issue of entropy “running low” is a straw man. About 256 bits of entropy are enough to get computationally secure numbers for a long, long time.

And the fun only starts here: how does /dev/random know how much entropy there is available to give out? Stay tuned!

There is literally a devices that transmits small portions of random numbers via one of the standard interfaces. And you may be prohibited to use any PRNG alghorhitms by spec of the device/firmware not because this PRNG has been proven as insecure.

Therefore I am inclined to think that it is necessary to continue to separate these two interfaces, although I always use urandom myself

[pbackus]

you may be prohibited to use any PRNG alghorhitms by spec of the device/firmware

In that case, you will have to bypass the Linux kernel entirely and write bare-metal code, since all random numbers generated by the Linux kernel come from a CSPRNG. I do not think it makes sense for an architecture-independent standard library to support this use-case.

[denizzzka]

I think the opposite:

1. OS can have any (proprietary) patch/module/driver about random and urandom sources
2. Architecture-independent standard library should not rely at all for a specific operating system internals.

So we need an architecture-independent library that should take random/urandom difference into account

[denizzzka]

What about Windows?

I just needed to somehow emphasize that we also need(?) numbers based on seeds too

[rikkimax]

Windows: it will be exactly the same. There isn't an infinite source of entropy that IC's have access to and CPU's are no different.

I suggest reading: Random Number Generators-Principles and Practices: A Guide for Engineers and Programmers
It was implemented by the person who implemented x86's random number instructions.

[pbackus]

For reference, the official source on how Windows's RNG works is "The Windows 10 random number generation infrastructure".

According to the summary on Wikipedia, there is a "root PRNG" in the kernel, which is used to seed per-process PRNGs, which are then used to generate numbers for the various RNG APIs.