Filtering: optional include/exclude `LogMatchProperties`

[albertlockett]

I was trying to use the new filtering code (#1341) to create filters that only include criteria for what I want to include (e.g., nothing should be excluded).

For example:

        let include = LogMatchProperties::new(
            MatchType::Strict,
            // include logs with any resource attributes
            vec![],
            // include all logs with any attributes
            vec![],

            // only include logs with severity_level = "ERROR"
            vec![
                "ERROR".into()
            ],
            None, // include all severity numbers
            vec![] // include all bodies
        );

        // don't exclude anything ...
        let exclude = LogMatchProperties::new(
            MatchType::Strict,
            vec![],
            vec![],
            vec![],
            None,
            vec![]
        );

        let filter = LogFilter::new(include, exclude, vec![]);

Unfortunately, this doesn't seem to give the desired behaviour. I think we have an opportunity to make the API easier to use, and also maybe improve performance.

I was thinking that LogFilter::new signature could be augmented to take Option<LogMatchProperties> for both include & exclude, then we could compute filters optionally for inlcude & exclude only if the the LogMatchProperties is Some. If they are both Some, only then do we need to and the filters together:

otel-arrow/rust/otel-arrow-rust/src/otap/filter/logs.rs

Lines 108 to 126 in 36d4a6b

	pub fn filter(&self, mut logs_payload: OtapArrowRecords) -> Result<OtapArrowRecords> {
	// we get the filters
	let (include_resource_attr_filter, include_log_record_filter, include_log_attr_filter) =
	self.include.create_filters(&logs_payload, false)?;
	let (exclude_resource_attr_filter, exclude_log_record_filter, exclude_log_attr_filter) =
	self.exclude.create_filters(&logs_payload, true)?;
	// combine the include and exclude filters
	let resource_attr_filter = arrow::compute::and_kleene(
	&include_resource_attr_filter,
	&exclude_resource_attr_filter,
	)
	.context(error::ColumnLengthMismatchSnafu)?;
	let log_record_filter =
	arrow::compute::and_kleene(&include_log_record_filter, &exclude_log_record_filter)
	.context(error::ColumnLengthMismatchSnafu)?;
	let log_attr_filter =
	arrow::compute::and_kleene(&include_log_attr_filter, &exclude_log_attr_filter)
	.context(error::ColumnLengthMismatchSnafu)?;

[albertlockett]

@c1ly maybe you could have a look when you have some time?