Why it matters
The small files problem is silent until it is catastrophic. A pipeline that produces one file per minute per source runs happily for months, until the day the NameNode heap crosses its allocation and garbage collection latency spikes. Then every RPC gets slower, then RPCs start timing out, and finally the NameNode restarts under memory pressure and comes back to the same problem in a few days.
Preventing small files is far cheaper than cleaning them up. A single line of pipeline code that batches output before writing is worth a hundred emergency ops calls later.
The architecture
Every file in HDFS consumes one inode entry in the NameNode's block map, plus one entry per block. For a small file with a single block, this is 300 bytes of NameNode heap. Multiply by tens of millions of files and the NameNode heap becomes dominated by metadata for files that hold almost no data.
DataNodes handle small files less badly because the storage is not the bottleneck; the block map on the NameNode is. But small files still hurt: block replication traffic for a million tiny blocks is dominated by protocol overhead, and disk seeks per byte read are much worse for many small files than for fewer large ones.
How it works end to end
The classic consolidation technique is Hadoop Archive (HAR). A HAR file is a single HDFS file that contains many logical files inside it, with an internal index. From the NameNode perspective it is one file with a few blocks. From the client perspective (using the har:// scheme) it is a virtual directory of files. HAR is simple and non-invasive; existing readers can be pointed at the archive with almost no code change.
SequenceFile is another approach: a binary key-value container where each key is a filename and each value is the file contents. This is friendlier to MapReduce and Spark because splits fall naturally on record boundaries. Avro and Parquet extend this idea with schema and columnar layout, and are the modern preferred choice for most data pipelines.
HBase turns the small file problem into a totally different pattern: HBase writes to memstores, flushes to HFiles, and periodically compacts HFiles into larger ones. Millions of small rows become a handful of HFiles per region. If your access pattern is by key, HBase can be the right home for what would otherwise be small files.