From the Association
universe:
KeyValueMap[Flatten @* List] @ Merge[# &] @ Map[GroupBy[First -> Rest], {a, b}]
{{1, 3, 5, 2}, {2, 6, 2, 1}, {3, 5, 6, 1, 2}, {4, 2, 3}}
Or
KeyValueMap[Flatten @* List] @ Merge[# &] @ MapApply[#1 -> {##2} &, Join[a, b]]
{{1, 3, 5, 2}, {2, 6, 2, 1}, {3, 5, 6, 1, 2}, {4, 2, 3}}