[WIP] Fix default allocator

[pkufool]

The purpose of this PR is to fix the default_context.cu so that we can use k2 without Pytorch. There are still some issues to be fixed.

• NvToolExt.h Not found error (if K2_ENABLE_NVTX=ON). I found that the cuda env was handle by Pytorch before, I am reading FindCUDA.cmake in Pytorch and try to extract some script there.
• About the cudaStream, I am still a little confued about how k2 or pytorch handle cuda streams, now I always allocate memory on the default stream (i.e. stream 0).
• The caching strategy (using default now, might need some tuning and benchmark).
• A global Caching allocator or each Context has its own allocator instance? Which one is better?

Even though this is a small fixes, allocator is the fundamental part, I may miss someting important, hope for your suggestions.

[pkufool]

Here, in Pytorch_Context.cu, we get the stream by c10::cuda::getCurrentCudaStream, I read the source code of Pytorch, if we don't setCurrentCudaStream the return stream is always the default stream (i.e. stream 0).

So, can we use the default stream 0 here? (0 might not be good here, we can add a macro some like kDefaultStream)

[pkufool]

Here, the allocator allocates memory with default stream 0, we can pass a stream to it as well.

[csukuangfj]

The purpose of this PR is to fix the default_context.cu so that we can use k2 without Pytorch

What is the use case? That is, how will we use k2 with cuda but without PyTorch?

[pkufool]

The purpose of this PR is to fix the default_context.cu so that we can use k2 without Pytorch

What is the use case? That is, how will we use k2 with cuda but without PyTorch?

For decoding purpose, if we use onnxruntime or tensorRT as the inference engine, I don't think we should depend on Pytorch at that time. The inference engine might have allocator, though, I am not sure.

[csukuangfj]

The inference engine might have allocator, though, I am not sure.

In that case, I would suggest creating another context that wraps the allocator from the given framework.

[pkufool]

The inference engine might have allocator, though, I am not sure.

In that case, I would suggest creating another context that wraps the allocator from the given framework.

Yes, we will see, just to fix the default_context.cu, it is not working now.