Skip to content

feat: array contains - #67

Open
mantasu wants to merge 3 commits into
ExpediaGroup:mainfrom
mantasu:feat/array-contains
Open

feat: array contains#67
mantasu wants to merge 3 commits into
ExpediaGroup:mainfrom
mantasu:feat/array-contains

Conversation

@mantasu

@mantasu mantasu commented Jul 31, 2026

Copy link
Copy Markdown
Member

Description

Adds an ArrayContains transformer and paired Keras layer that check whether a scalar value is contained in an array feature, outputting 1.0 if present and 0.0 otherwise.

Keras Layer Checklist

  • The new Keras layer extends BaseLayer
  • The _call method has been implemented in the new layer.
  • The compatible_dtypes property is defined in the new layer.
  • The new layer is decorated with @tf.keras.utils.register_keras_serializable(package=kamae.__name__).
  • The new layer takes a name, input_dtype, and output_dtype as arguments to the constructor and that this is passed to the super constructor.
  • The Keras layer is serializable. I have implemented the get_config method.
  • There are unit tests of the new layer.
  • There is a specific test of layer serialisation added here.
  • The new layer is imported in the init.py file in the layers directory.

Spark Transformer/Estimator Checklist

  • The new Spark Transformer extends BaseTransformer.
  • If the new transform needs a fit method, a Spark Estimator has been implemented that extends BaseEstimator.
  • The instructions in the above docs page have been followed for the __init__ and setParams methods.
  • The transformer uses one of the input/output mixin classes from base.py.
  • If the new transformer requires more parameters that would need to be serialised to the Spark ML pipeline, there is a implemented parameter class by extending the Params class here.
  • The compatible_dtypes property has been implemented to specify the input/output data types that my transformer/estimator supports.
  • A Keras subclassed layer is returned in the transformer's get_tf_layer method.
  • There are unit tests of the new transform. In particular, there are parity tests between the Spark and Keras implementations.
  • The new transformer/estimator is imported in the init.py file in the transformers/estimators directory.

Readme Checklist

  • There is a new entry (alphabetical order) in the README table describing the new layer/transformer

@mantasu
mantasu requested a review from a team as a code owner July 31, 2026 12:22
@mantasu
mantasu requested review from ddonghi and jacobjwood July 31, 2026 12:22
@mantasu

mantasu commented Jul 31, 2026

Copy link
Copy Markdown
Member Author

Could I please be also given write access so I wouldn't need to create the fork? 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant