Hi Kevin, thanks for reply. I could hardly imagine an example of incomplete record. The mapper is very simple, just reading line by line as Strings, splitting the line by tab, and outputting a Text Pair for sort and secondary sort. If there were incomplete record, there should be an error and the only place could happen is the tab splitting stage. I use LZO codec compressing the Mapper output and Reducer output. There is one issue which I think might be the reason. The files in the log has the -- Shi Yu
↧