There is still a header to send. Talking about SVG that are on average 465 bytes, a 867 bytes header (as sent by GitHub[0]) is not negligible. HTTP compresses headers though, so according to [1] we can expect ~30% header transferred size/actual size ratio which brings header transferred size to ~260 bytes. Still a 56% overhead for the average icon from this collection.
That depends on the transport, this is one of the things that HTTP/2 (and HTTP/3) aim to fix by downloading them in parallel in a single connection.
Watch the devtools network tab: http://www.http2demo.io/