Skip to content

Even Faster SumOption!

Compare
Choose a tag to compare
@johnynek johnynek released this 02 Dec 22:53
· 489 commits to develop since this release
3c0d3a5

The main new feature of this release is a faster (benchmarked!) implementation of tuple and product semigroup sumOptions. This means if you are aggregating on scalding or spark, you should see a significant (~ 2x faster).

There is a new Set membership monoid called SetDiff. It can model adding and removing from sets (which can be useful for applications in summingbird).

We have an exponential histogram Fold, which is an approximate data-structure that can tell you approximate counts over sliding windows (see #568). A future work will add a monoid for this type, however when possible, using the Fold is better since it has better error properties.

Lastly, there are many new docs.

Huge thanks to @sritchie who was the main contributor to this release.

changelog:

  • Add SetDiff data structure to algebird-core: #555
  • Add Ring[BigDecimal], modeled after Ring[BigInt]: #553
  • "Exponential Histogram" sliding window counter implementation added to algebird-core as ExpHist: #568
  • improve HLLSeries performance: #575
  • Add a microsite at https://twitter.github.io/algebird via the sbt-microsites plugin, along with docs for all typeclasses and data structures: #576
  • Adds lots of scalacheck Arbitrary and Gen instances to algebird-test, under com.twitter.algebird.scalacheck.{ gen, arbitrary }: #579
  • Add Monoid[Max[Vector[T]]], Monoid[Max[Stream[T]]]: #579
  • Add FirstAggregator and LastAggregator, and docs and API / perf improvements for First, Last, Min, Max: #579
  • Add LawsEquiv versions of all laws: #584
  • Deprecates broken group/ring for Future/Try: #584
  • Add metricsLaws[T] to BaseProperties in algebird-test: #584
  • Modify generated Tuple2Monoid, etc to extend TupleNSemigroup, giving subclasses access to efficient sumOption: #585
  • optimize Generated{Abstract,Product}Algebra.sumOption with benchmarking #591
  • Add an efficient sumOption, +, -, methods and docs to AveragedValue: #589