Feature: run only selected tests

RpmGrill: allow filtering by test

Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1202633

Changes to user interface

Listing all plugins and its tests

rpmgrill --list-plugin-tests

plugin a   : description of plugin
  - test   : foobar
  - test 2 : does something else 
plugin b
  - test 

list tests of some plugins

rpmgrill --plugin x,y,z --list-tests

will list all tests of plugin x, y, and z

plugin x   : description of plugin
  - test a  : foobar
  - test b  : does something else 
plugin y
  - test
  - ...
 

Run: only some plugins

rpmgrill --run x,y,z --exclude a,b,z

will run all tests of x,y plugin

NOTE: it will not run z since it is the exclude list

Run: only some tests of some plugins

rpmgrill --run x:t1,t2,t3,y:,z:t2 
      # -- is same as --
rpmgrill --run x:t1,t2,t3  --run y --run z:t2

will run - t1, t2, t3 tests of plugin x - all tests of plugin y - t2 tests of plugin z

NOTE: when user filters by test, (s)he needs to use y: to indicate y is plugin and not a test in plugin x

What needs to be changed

How it works currently

RpmGrill uses Module::Pluggable to load all plugins under lib/Rpm/Grill/Plugin/*.pm rpmgrill script calls Rpm::Grill::plugins which lists all available plugins. it then invokes $grill->invoke_plugin($plugin) which in turn calls $plugin->analyze()

Inside a plugin, analyze is the main method, which calls sub-methods and deep within the call tree is where the actual analysis is done.

Changes needed to support sub-test filtering

To support sub-test filtering, we need to be able to run individual sub-tests, so the plugins should start returning a list of TestCodes it supports and be able to run it.

approach 1: call any test

@tests_in_plugin = $plugin->tests();
$plugin->test('FooBar')->run()

instead of $plugin->analyze which runs all tests - currently

--- or ---

approach 2: pass sub-tests to run and let plugin do the work

$plugin->analyze(List, of, TestCodes, To, Run); 

and it is up to plugin to decide how to use the list of TestCodes passed and run only those. This may result in faster runs since the plugin gets to optimise common method like: setup, cleanup and just have one outer loop.

Question: wouldn't every plugin then have repeated

Preferred approach

Both approaches involve changing every plugin but a nice thing about the first one is that, Grill.pm which runs each plugin can use the Module::Pluggable::can and test if tests sub exist and use that to run individual tests.

### pseudo-code 
Grill::invoke_plugin($plugin, @disabled_tests) {
   if ( $plugin->can('tests') ) {
        # filter out disabled tests
        
        foreach $test (filter_out_disabled(@disabled_tests)) {
            $plugin->test($test)->setup()
            $plugin->test($test)->run()
            $plugin->test($test)->cleanup()
        }
   } else {
       $plugin->analyse;
   }
}

Questions/Doubts/Conclusion

Does this all make sense?

Isn't all this equal to separating out each test in a plugin into another plugin?