LogRecord and SpanByte changes #1186
Conversation
- Move CountdownWrapper to its own file - Rename Lockable* to Transactional* - Rename Transient* to Ephemeral*
Other work to get CreateNewRecordRMW clean
…ns first pass done)
| var storeWrapper = clusterProvider.storeWrapper; | ||
| var transientObjectIdMap = storeWrapper.store.Log.TransientObjectIdMap; | ||
|
|
||
| // Use try/finally instead of "using" because we don't want the boxing that an interface call would entail. Double-Dispose() is OK for DiskLogRecord. |
There was a problem hiding this comment.
using does not box its argument, it'll do a constrained virtual call on a value type.
To illustrate:
S() will compile to something like:
IL_0000 ldloca.s 00
IL_0002 initobj FooStruct
IL_0008 leave.s IL_0018
IL_000A ldloca.s 00
IL_000C constrained. FooStruct
IL_0012 callvirt IDisposable.Dispose ()
IL_0017 endfinally
IL_0018 ret
O() to:
IL_0000 newobj FooObj..ctor
IL_0005 stloc.0
IL_0006 leave.s IL_0012
IL_0008 ldloc.0
IL_0009 brfalse.s IL_0011
IL_000B ldloc.0
IL_000C callvirt IDisposable.Dispose ()
IL_0011 endfinally
IL_0012 ret
Note the absence of any box ops in S() - the only real difference is the constrained prefix.
libs/server/Storage/Session/Common/ArrayKeyIterationFunctions.cs
Outdated
Show resolved
Hide resolved
libs/storage/Tsavorite/cs/src/core/Allocator/ObjectAllocatorImpl.cs
Outdated
Show resolved
Hide resolved
| internal readonly bool IsSet => SegmentSizeBits != 0; | ||
| internal readonly bool HasData => word != 0 && word != NotSet; | ||
|
|
||
| public ObjectLogFilePositionInfo() |
There was a problem hiding this comment.
Empty struct constructors are tricky - they aren't invoked for default, or when a struct is implicitly initialized (like when it's a field). That is, new ObjectLogFilePositionInfo().word != default(ObjectLogFilePositionInfo).word.
Makes me a bit nervous.
| firstRead = true; | ||
| } | ||
|
|
||
| // Note: We may be sending to multiple replicas, so cannot serialize LogRecords directly to the network buffer |
There was a problem hiding this comment.
Every connection has a distinct network buffer, so we can technically serialize it directly to the network buffer.
Is there another reason why we cannot write the key,value pair directly to the network buffer as previously?
| /// </summary> | ||
| public PinnedSpanByte GetAvailableNetworkBufferSpan() => PinnedSpanByte.FromPinnedPointer(curr, (int)(end - curr)); | ||
|
|
||
| public void IncrementRecordDirect(int size) |
There was a problem hiding this comment.
Looks like this is not used anywhere? Do we need it?
| db.MainStore.Reset(); | ||
| if (db.ObjectStore?.Log.TailAddress > 64) | ||
| db.ObjectStore?.Reset(); | ||
| if (db.Store.Log.TailAddress > 64) |
There was a problem hiding this comment.
Should this be PageHeader.Size?
libs/server/Storage/Session/Common/ArrayKeyIterationFunctions.cs
Outdated
Show resolved
Hide resolved
| var valueSB = scratchBufferBuilder.FormatScratch(sizeof(long), value).SpanByte; | ||
| valueSB.ExtraMetadata = DateTimeOffset.UtcNow.Ticks + expiry.Ticks; | ||
| return SET(ref _key, ref valueSB, ref context); | ||
| var input = new RawStringInput(RespCommand.APPEND, ref parseState, arg1: DateTimeOffset.UtcNow.Ticks + expiry.Ticks); |
…threshold configuration option
…object-log segments); PendingContext is no longer itself a TSourceLogRecord;
This PR converts Garnet to use a new ObjectAllocator with a revised in-memory and on-disk format that will use only a single log file, change ISessionFunctions to be (Disk)LogRecord-based and propagate this to operations such as Compaction and Migration, remove TKey and TValue from TsavoriteKV, and many related refactors.
Highlights:
(ReadOnly)Span<byte>extensions.ReadOnlySpan<byte>at the StorageSession and below, including Tsavorite. There are no longer any byte[] keys. The TKey type argument is gone from Tsavorite and Garnet.(ReadOnly)Span<byte>or objects implementing IHeapObject. TValue has been removed from TsavoriteKV.Major remaining tasks to be addressed in follow-on PRs:
(ReadOnly)Span<byte>path by replacing byte* usage (such as was done for Utility.HashBytes).