Skip to content

Commit 3549fae

Browse files
rhou1rhou1
authored
Test Framework changes to enable some order-by clause verification to work for JSON strings. (#250)
1 parent 0d8dfda commit 3549fae

File tree

3 files changed

+375
-16
lines changed

3 files changed

+375
-16
lines changed

‎README.md‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,127 @@ Example:
131131
Use this option to provide the usage of the command, which includes additional options
132132
</code></pre>
133133

134+
### Special handling for order-by tests
135+
136+
The Test Framework is able to verify the order in which rows are returned for
137+
some SQL statements that have an order-by clause. If the columns in the order-by
138+
clause are also projected (present) in the output, then the Test Framework may
139+
be able to check the order of the rows that are returned. Here are some
140+
requirements for a query to be validated by the Test Framework:
141+
142+
1) All columns/fields in the order-by clause must appear in the projection list
143+
2) Expressions cannot be used in the order-by clause. Things like
144+
"order by column.field[2]". The [2] indicates an expression which is the
145+
third element in the field array.
146+
3) Referencing a field within a json string in a column is more complicated.
147+
Most cases are supported. Some cases may not work.
148+
4) If a query references more than one table, then use aliases for each column
149+
in the projection list, and reference these aliases in the order-by clause.
150+
Using aliases is a good practice in general when verifying an order-by clause.
151+
5) The order-by clause cannot be followed by another SQL operation except for
152+
limit. If the order-by clause is followed by an offset or collate, for
153+
example, it might not work.
154+
155+
Here are some queries that can be validated:
156+
157+
1) select id, gbyi from `complex.json` order by id limit 10;
158+
159+
+-----+-------+
160+
| id | gbyi |
161+
+-----+-------+
162+
| 1 | 1 |
163+
| 2 | 2 |
164+
| 3 | 3 |
165+
| 4 | 4 |
166+
| 5 | 5 |
167+
| 6 | 6 |
168+
| 7 | 7 |
169+
| 8 | 8 |
170+
| 9 | 9 |
171+
| 10 | 10 |
172+
+-----+-------+
173+
174+
This query can be validated because the results are ordered by the "id" column,
175+
and the "id" column is projected in the output. The Test Framework can
176+
examine the output and verify that the rows are in order.
177+
178+
2) select * from
179+
(select d.uid uid, flatten(d.events) evnts from `data.json` d order by d.uid) s
180+
order by s.evnts.event_time, s.evnts.campaign_id;
181+
182+
+-------+--------------------------------------------------------------------------------------------------+
183+
| uid | evnts |
184+
+-------+--------------------------------------------------------------------------------------------------+
185+
| 1 | {"event_name":"e1_name","event_time":1000000,"type":"cmpgn1"} |
186+
| 1 | {"event_name":"e2_name","event_time":2000000,"type":"cmpgn4","evnt_id":"e2","campaign_id":"c1"} |
187+
| null | {"event_name":"e2_name","event_time":2000000,"type":"cmpgn4","evnt_id":"e2","campaign_id":"c1"} |
188+
| 1 | {"event_name":"e3_name","event_time":3000000,"type":"cmpgn1","evnt_id":"e3","campaign_id":"c1"} |
189+
| null | {"event_name":"e3_name","event_time":3000000,"type":"cmpgn1","evnt_id":"e3","campaign_id":"c1"} |
190+
| null | {"event_name":"e4_name","event_time":4000000,"type":"cmpgn1","evnt_id":"e4","campaign_id":"c1"} |
191+
| 1 | {"event_name":"e5_name","event_time":5000000,"type":"cmpgn3","evnt_id":"e5","campaign_id":"c2"} |
192+
| null | {"event_time":6000000,"type":"cmpgn9","evnt_id":"e6","campaign_id":"c1"} |
193+
| 1 | {"event_name":"e6_name","event_time":6000000,"type":"cmpgn9","evnt_id":"e6"} |
194+
| null | {"event_name":"e7_name","event_time":7000000,"type":"cmpgn3","evnt_id":"e7","campaign_id":"c1"} |
195+
| null | {"event_name":"e8_name","event_time":8000000,"type":"null","evnt_id":"e8","campaign_id":"c2"} |
196+
| 1 | {"event_name":"e8_name","event_time":8000000,"type":"cmpgn2","evnt_id":"e8","campaign_id":"c2"} |
197+
| null | {"event_time":9000000,"type":"cmpgn4","evnt_id":"e9","campaign_id":"c2"} |
198+
| 1 | {"event_name":"e9_name","event_time":9000000,"type":"cmpgn4","evnt_id":"e9"} |
199+
| 1 | {"event_name":"e7_name","type":"cmpgn3","evnt_id":"e7","campaign_id":"c1"} |
200+
| 1 | {"event_name":"e4_name","type":"cmpgn1","evnt_id":"e4","campaign_id":"c1"} |
201+
| null | {"event_name":"e1_name","type":"cmpgn9","campaign_id":"c1"} |
202+
| null | {"event_name":"e5_name","type":"cmpgn2","evnt_id":"e5","campaign_id":"c2"} |
203+
+-------+--------------------------------------------------------------------------------------------------+
204+
205+
This query can be validated because the results are ordered by the data in the
206+
"evnts" column, and the "evnts" column is projected in the output. The Test
207+
Framework can parse the JSON string in the "evnts" column and examine the
208+
event_time and campaign_id values.
209+
210+
211+
These queries cannot be validated:
212+
213+
1) select t.gbyt, t.id, t.ooa[0].`in` zeroin, t.ooa[1].fl.f1 flf1, t.ooa[1].fl.f2 flf2, t.ooa[1].`in` onein, t.ooa[2].a.aa.aaa, t.ooa[2].b.bb.bbb, t.ooa[2].c.cc.ccc from `complex.json` t where t.ooa[2].b.bb.bbb is not null order by t.ooa[2].c.cc.ccc limit 10;
214+
215+
+-------+--------+---------+-------------+-----------+--------+------------+------------+------------+
216+
| gbyt | id | zeroin | flf1 | flf2 | onein | EXPR$6 | EXPR$7 | EXPR$8 |
217+
+-------+--------+---------+-------------+-----------+--------+------------+------------+------------+
218+
| aaa | 10 | null | null | null | 10 | aaa 10 | bbb 10 | ccc 10 |
219+
| ooos | 1000 | null | null | null | 1000 | aaa 1000 | bbb 1000 | ccc 1000 |
220+
| nul | 10002 | null | null | null | 10002 | aaa 10002 | bbb 10002 | ccc 10002 |
221+
| sba | 10003 | 10003 | 10003.6789 | 154351.0 | 10003 | aaa 10003 | bbb 10003 | ccc 10003 |
222+
| str | 10008 | 10008 | null | null | 10008 | aaa 10008 | bbb 10008 | ccc 10008 |
223+
| fl | 10009 | 10009 | null | null | 10009 | aaa 10009 | bbb 10009 | ccc 10009 |
224+
| saa | 1001 | null | 1001.6789 | 64331.0 | 1001 | aaa 1001 | bbb 1001 | ccc 1001 |
225+
| soa | 10023 | null | null | null | 10023 | aaa 10023 | bbb 10023 | ccc 10023 |
226+
| nul | 10028 | null | 10028.6789 | 154601.0 | 10028 | aaa 10028 | bbb 10028 | ccc 10028 |
227+
| ooos | 10029 | null | 10029.6789 | 154611.0 | 10029 | aaa 10029 | bbb 10029 | ccc 10029 |
228+
+-------+--------+---------+-------------+-----------+--------+------------+------------+------------+
229+
230+
This query cannot be validated because the order-by has an expression,
231+
"t.ooa[2].c.cc.ccc". The Test Framework cannot evaluate the array reference
232+
"ooa[2]".
233+
234+
2) select id from `complex.json` order by gbyi limit 10;
235+
236+
+------+
237+
| id |
238+
+------+
239+
| 106 |
240+
| 121 |
241+
| 91 |
242+
| 46 |
243+
| 61 |
244+
| 31 |
245+
| 1 |
246+
| 76 |
247+
| 16 |
248+
| 136 |
249+
+------+
250+
251+
This query cannot be validated because the "gbyi" column is not projected so
252+
the Test Framework cannot determine what the order should be.
253+
254+
134255
## Authors
135256

136257
[Zhiyong](https://github.com/zhiyongliu)
@@ -146,3 +267,4 @@ Example:
146267
[Jacques](https://github.com/jacques-n)
147268
[Jason](https://github.com/jaltekruse)
148269
[Sudheesh](https://github.com/sudheeshkatkam)
270+
[Robert](https://github.com/rhou1)

0 commit comments

Comments
 (0)