Bug Report
We use a round-robin way to request the TSO server with FindGroupByKeyspaceID gRPC call.
|
func (c *tsoServiceDiscovery) updateMember() error { |
|
// The keyspace membership or the primary serving address of the keyspace group, to which this |
|
// keyspace belongs, might have been changed. We need to query tso servers to get the latest info. |
|
tsoServerAddr, err := c.getTSOServer(c.apiSvcDiscovery) |
|
if err != nil { |
|
log.Error("[tso] failed to get tso server", errs.ZapError(err)) |
|
return err |
|
} |
|
|
|
keyspaceID := c.GetKeyspaceID() |
|
var keyspaceGroup *tsopb.KeyspaceGroup |
|
if len(tsoServerAddr) > 0 { |
|
keyspaceGroup, err = c.findGroupByKeyspaceID(keyspaceID, tsoServerAddr, updateMemberTimeout) |
|
if err != nil { |
|
if c.tsoServerDiscovery.countFailure() { |
|
log.Error("[tso] failed to find the keyspace group", |
|
zap.Uint32("keyspace-id-in-request", keyspaceID), |
|
zap.String("tso-server-addr", tsoServerAddr), |
|
errs.ZapError(err)) |
|
} |
|
return err |
|
} |
|
c.tsoServerDiscovery.resetFailure() |
|
} else { |
However, if the requested TSO server satisfies this kind of case, it may cause the timestamp to fallback.
- TSO node A initialized group 1.
- TSO node B has not initialized group 1 yet.
If we do updateMember quickly while requesting the TSO, it may cause the keyspace group in the request to change between the default group and group 1, resulting in the timestamp fallback.
