It's strange to reach the memory limit with such a small raster/grid. Even a 32 bits program would deal with it fine. Do you have options regarding memory limitation on Qt Creator?
By the way, does it work in release mode? (assert is a debug-only feature).
Daniel, CloudCompare admin