How should we represent distribution-generic array formals?

[DanilaFe]

Today, formals written with array type expressions are distribution-generic. Thus, the following code works:

use BlockDist;

var A: [1..10] int;
var B = blockDist.createArray(1..10, int);

proc foo(arr: [] int) {
  compilerWarning(arr.type : string);
}

foo(A);
foo(B);

It also doesn't cause a coercion of B to a default-rectangular array. Instead, we end up with two instantiations (one for default-rectangular and one block-distributed). Thus, the output of the compiler here is:

distgen-1.chpl:1: In module 'distgen-1':
distgen-1.chpl:10: warning: [domain(1,int(64),one)] int(64)
distgen-1.chpl:11: warning: [BlockDom(1,int(64),one,unmanaged DefaultDist)] int(64)

This is true even if arr is given a fully concrete-looking type expression like [1..10] int.

proc foo(arr: [1..10] int) { /* I behave the same */ }

In general, this works fine. However, there are two notable exceptions that I am aware of today:

out formals

For out formals, array expressions seem to be concrete in some cases. The following makes arr distribution-generic, and initializes A with a block-distributed array:

use BlockDist;
var A;
proc foo(out arr: [] int) {
  arr = blockDist.createArray(1..10, int);
}

foo(A);
compilerWarning(A.type:string, 0); // [BlockDist...] int

However, specifying the domain expression makes the result be a concrete, non-distributed array.

use BlockDist;
var A;
proc foo(out arr: [1..10] int) {
  arr = blockDist.createArray(1..10, int);
}

foo(A);
compilerWarning(A.type:string, 0); // [domain(...)] int, aka default-rectangular

One curious way to break the above is to mix out and non-out formals:

proc bar(out A: [1..10] int, B: [1..10] int) { A = B; }
var A;
bar(A, blockDist.createArray(1..10, int)) {}

Even though the A and B formals have identical type expressions, one accepts a block-distributed array, and ends up performing a many-to-one copy to initialize a (non-distributed) A. The variable A ends up being default-rectangular.

@bradcray mentions that this is roughly to match the behavior of return type expressions, since out formals are notionally another way for a function to return data. Type expressions that specify return types of functions are treated as concrete.

Compiler-generated initializers

A similar issue arises when we use records with array fields. Take in particular:

record R {
  var A: [1..10] int;
}

This is a concrete record with a default-rectangular field A. At the surface, this is fine; only array formals are generic, and this isn't a formal. However, even though the compiler-generated initializer notionally looks like this:

proc R.init(A: [1..10] int) { this.A = A; } /* compiler-generated */

The A formal here is not generic. Thus, when trying to invoke the default initializer for R with a block-distributed array, we get a compiler error.

blockarraytorecordfield.chpl:1: In module 'blockarraytorecordfield':
blockarraytorecordfield.chpl:10: error: unresolved call 'R.init([BlockDom(1,int(64),one,unmanaged DefaultDist)] int(64))'
blockarraytorecordfield.chpl:5: note: this candidate did not match: init(this: R, A) 
blockarraytorecordfield.chpl:10: note: because actual argument #1 with type '[BlockDom(1,int(64),one,unmanaged DefaultDist)] int(64)'
blockarraytorecordfield.chpl:7: note: is passed to formal 'in A: [domain(1,int(64),one)] int(64)'

This might be reasonable as well (the user is trying to force conversion as part of an initializer, which can be confusing), but if the user were to write this initializer explicitly, it would work!

proc R.init(A: [1..10] int) { this.A = A; } /* user-written */

As in the weird out-copy case, this would allow the argument A to be block-distributed, and perform a many-to-one copy when initializing this.A.

Should array formals with generic management be explicitly marked as such?

There's an argument to be made that we have moved towards explicitly marking generic type expressions. The recent push towards '?' for generic types in formals and field type expression is an instance of this ethos. However, arrays today are implicitly generic, so it makes sense to require that the user marks them as such. One idea was an explicit dmapped ? syntax in a formal type expression:

proc foo(A: [{1..1} dmapped ?] int) {} /* accepts generics */
proc foo(A: [{1..1}] int) {} /* accepts default-rectangular arrays */

This change (or any change to the genericity of array formals) would be breaking and ought to be gated behind an edition or major release.

A counter-argument might be that making array expressions implicitly generic makes writing Chapel code more natural, and contributes to the feeling of distributed arrays (etc) as first-class citizens of the language. "A distributed array is just an array, not some special type!". Forcing explicit genericity and type syntaxes might reduce the advantages of a unified array type compare to special types per-distribution etc.

[e-kayrakli]

I can't help but think that this is somewhat related to #23328. While there are specific challenges this issue brings up, I believe that there is an overarching challenge by distributions not being as first-class as domains and arrays.