Either by lack of existing solutions for gRPC caching or by need for a non-trivial logic you might require to use an application cache for your responses.
Without any extra support, with a bit of effort you can implement middleware level caching just with existing middleware interface and its
next function, allowing you to skip the following handlers.
This simple middleware is a good start: we effectively skip the handlers, can manage TTL and can combine with any custom logic for the matching. For example we can hook the cache middleware after authentication middleware and create per-user cache with prepared data about user from the call.
Assume we have other than in-memory cache and we need to serialize the instances into some form and back. That is why there are
protoToCached functions. This is the troublesome part: doing this serialization generically might be challenging (and could be potentially solvable only via custom solution for each RPC/message) and it might not be "cheap" (cache hits should be blistering fast and not perform double encoding
cache-to-proto by you and
proto-to-wire by gRPC).
That's why ProtoCat has a mechanism to remove both of the problems: generic approach for all RPCs without mapping functions and avoid double encoding.
If you don't care too much about the details, you can just use
It does not matter what implementation you have, any sync/async key-value storage that can handle buffers is fine.
It is crucial that you return the binary data in buffer as you received them. Be mindful of any encoding changes or data truncate that might happen in your cache. If you fail to return a buffer with the same contents as provided by the set function, you will likely send invalid data on the wire. This will likely result in your client receiving "valid" responses but will crash on the data decoding.
In case you don't want to cache some RPC, your
hash must return falsy value. In that case the middleware will not try to retrieve value from cache nor will save the result. Implement any logic your heart desires in
hash function. You can read the RPC info, client metadata, data from previous middlewares etc.
It is all up to you. If you want different TTL for different calls, set the TTL (or delete the record in your cache) yourself. If your
cache.get returns falsy value, we consider it to be a cache miss for whatever reason (cold start, ttl, etc.). Same goes for cache purging. ProtoCat never needs to clear your cache, so there is no API in the interface for that and you manage it on your own.
ProtoCat uses the "lowest" available public API of
grpc-js to hook handlers, that is
server.register. This allows it to override serialization methods. This way the standard protobuf message to wire function can be overriden to do noop if ProtoCat tells it buffer caching has been used and it skips the default transformation of message to binary.
This is also the reasoning behind how we implemented the generic method. Now that the wire format can be used directly, it is also used as a serialization format that goes to cache storage. That is why it is so crucial to respect the buffer contract and not to tamper with the data in any way. Under normal cricumstances, sending an unserielizable input down gRPC would cause an error on the server, during the encoding. With caching middleware these runtime checks are skipped.
The middleware creator should satisfy your needs, but if you would like to use this mechanism without the middleware helper, it uses public API of the ProtoCat's call context. See the implementation for details.
Cache middleware is skipped for all call types but
unary. The underlying mechanism works for client streaming as well, but is far less powerful, since hashing function does not receive any client messages and must decide without or read them manually from stream.
At this point I am convinced that it does not make sense to cache streaming calls, since they should be used either for live data or as an event communication, none of which makes sense to be cached. If you find a use case for streaming caching open an issue! 😺