Function tokens_cmpeq 

pub fn tokens_cmpeq(
    a: TokenStream,
    b: TokenStream,
    cmp_span: Span,
) -> Result<Equality>

Compares two TokenStreams for “equivalence”

We intend that two TokenStreams count as “equivalent” if they mean the same thing to the compiler, modulo any differences in spans.

We also disregard spacing. This is not 100% justifiable but I think there are no token sequences differing only in spacing which are both valid and which differ in meaning.

Why ?!

< < and << demonstrate that it is not possible to provide a fully correct and coherent equality function on Rust tokens, without knowing the parsing context:

In places where << is a shift operator, < < is not legal. But in places where << introduces two lots of generics, << means the same.

I think a function which treats < < and << as equal is more useful than one that doesn’t, because it will DTRT for types.

None-delimited Groups

We flatten these

This is necessary, because otherwise apparently-identical pieces of code count as different.

This does mean that two things which are approx_equal can be expressions with different values!

But, the Rust grammar for types doesn’t permit ambiguity, so the type equality guarantee of approx_equal is preserved.