How to Store Events in SimpleFeed

Understanding the Score and the Value

From the library perspective, the only thing you can store as a value: is of data-type String, which is necessarily associated with a score of type Double. You can provide a custom score in the at: argument of #store, but without this argument the current timestamp will be used by the default.

• While the most common use for the score is as a (current) timestamp, other uses are possible.

• The value is always custom — based on the implementation. It represents your way of serializing the events.

This is because how you store your events is up to you, the developer, to decide.

Having said that, here are some considerations:

Considerations for Event Serialization

• the less you store the more a single-instance Redis backend can scale up without having to be sharded

• the less you store the faster network interactions will be with the feed, and less likely it will become a point of contention

• the less you store in SimpleFeed, the more you, probably, will need to fetch in addition to what SimpleFeed already returns, i.e. in order to show the user their activity full with the name of the authors, comments, etc.

With that in mind, imagine that we are building an application where most objects in the system have a database identifier field. If we must store various types of models as events, we can choose a custom serialization scheme where we reserve one or two letters for the model name, then a dot, then the database ID — as a long integer that's converted to a base of 64 string.

For example,

require 'base64'

module CompressedID
  def compressed_id
    @compressed_id ||= self.class.compress_id(id)
  end

  module CompressedClassMethods
    UNPACK_MAPPING = {
      'C' => 8,
      'S' => 16,
      'L' => 32,
      'Q' => 64
    }

    PACK_MAPPING = UNPACK_MAPPING.invert

    def pack_from_int(value)
      len = value.bit_length
      PACK_MAPPING.keys.sort.each do |bits|
        if len < bits
          return PACK_MAPPING[bits]
        end
      end
      raise ArgumentError, "Argument #{value} needs #{len} bits, which exceeds maximum of 64"
    end

    def class_marker
      (self.is_a?(Class) ? self.name : self.class.name).split('::').last[0..1].upcase
    end

    def compress_id(id)
      pack_marker = pack_from_int(id)
      value = class_marker + pack_marker + Base64.urlsafe_encode64(Array(id).pack("#{pack_marker}*"))
      value[0..-2]
    end

    def decompress_id(encoded)
      pack_marker = encoded[2..2]
      string = encoded[3..-1] + '='
      if encoded.start_with?(class_marker)
        Base64.urlsafe_decode64(string).unpack("#{pack_marker}*").first
      end
    end
  end

  def self.included(base)
    base.extend(::CompressedID::CompressedClassMethods)
  end
end

# =============================================================================

module Example
  class Post
    include CompressedID
    attr_accessor :id, :body
    def initialize(id, body = nil)
      self.id = id
      self.body = body
    end
  end
end

# =============================================================================

require 'rspec'
require 'rspec/its'

RSpec.configuration.color = true
RSpec.configuration.add_formatter 'progress'

RSpec.describe Example::Post do
  context 'class methods' do
    subject { described_class }
    its(:class_marker) { should eq 'PO' }
    it '#compress_id' do
      expect(subject.compress_id(1)).to eq('POCAQ=')
    end
    it '#decompress_id' do
      expect(subject.decompress_id('POCAQ=')).to eq(1)
    end
  end

  shared_examples :compressable do |id, compressed_id|
    before do
      expect(compressed_id).to be_kind_of(String)
      expect(id).to be_kind_of(Numeric)
    end
    subject(:post) { Example::Post.new(id, 'test') }
    its(:id) { should eq(id) }
    its(:compressed_id) { should eq(compressed_id) }
    it 'should decompress using a class method' do
      expect(post.class.decompress_id(compressed_id)).to eq(post.id)
    end
  end

  TEST_CASES = [
    [1,                     'POCAQ='],
    [100_000_000,           'POLAOH1BQ='],
    [23_909_809_809,        'POQkb4ikQUAAAA'],
    [1_012_938_091_823_012, 'POQpH8kB0OZAwA']
  ]

  TEST_CASES.each do |use_case|
    describe "#{use_case.inspect}" do
      it_should_behave_like :compressable, use_case[0], use_case[1]
    end
  end

  context 'numbers larger than 64-bit' do
    let(:id) { 12445345342342345435455 }
    it('uses more than 64 bits') { expect(id.bit_length).to be > 64 }
    it 'throws ArgumentError' do
      expect { Example::Post.new(id, 'test').compressed_id }.to raise_error(ArgumentError)
    end
  end
end

This spec, of course, passes:

 1 ❯ be rspec .
.................

Finished in 0.00604 seconds (files took 0.07437 seconds to load)
17 examples, 0 failures

Restoring Objects from Serialized Format

Then use #compressed_id to save as a Value, in #store(value: ...). Upon retrieving a value from the SimpleFeed, you would read the result and instantiate an object by calling eg.

class Post
  def self.news_feed_for(user)
    activity = SimpleFeed.feed(:news).activity(user.id)
    events = activity.paginate(page: 1)
    events.map do |event|
      self.new(self.decompress_id(event))
    end
  end
end