The fastest way to copy something over to your phone (onto the memory card) is by using a USB2 capable card reader/writer.

I don't know if the SE phones are USB1 or USB2 capable, but it might explain the slow transfer speed if it's only USB1.

USB1 allows a maximum transfer speed of 12Mbps which is actually around 1.5Mb/s, whereas USB2 allows up to 480Mbps (~60Mb/s). However, of course this is also limited by the speed it's possible to write to the card at. I don't have any benchmarks for the performance of M2 memory cards, but memory cards produced by Sandisk are usually quite slow in my experience unless they are the 'Ultra' series.