For motion boundary coding it is natural to try to capture the local orientations of mo-tion edges by emulating the static-image HOG descriptors. The simplest approachis to treat the two flow components $Ix,Iy$ as independent ‘images’, take their localgradients separately, find the corresponding gradient magnitudes and orientations, anduse these as weighted votes into local orientation histograms in the same way as for thestandard gray scale HOG. We call this family of schemes Motion Boundary Histograms(MBH). A separate histogram can be built for each flow component, or thetwo channels can be combined, e.g. by the winner-takes-all voting method used to han-dle color channels in.

Source, Image Source: Human Detection Using Oriented Histograms of Flow and Appearance [PDF]