How to: convert line endings to UNIX with dos2unix command on your unix machine when importing data flat files.
About
Files created on windows systems have different line endings to those made on Unix/Linus systems. Windows uses a combination of Carrige Return (CR) and Line Feed (LF) while Unix uses only LF.
In may cases reading a file created on a Windows system from a Unix system is fine becuase Windows uses both line endings. However this can cause issues when reading in data files since the Unix system will use the LF to identify the endof line but leave the CR character in place. This results in the last datafield containing a CR cahracter at the end of the value. Not often an issue where the value is formatted as a numeric field but potentially troublesome if the value is formatted as a character.
Code
Below is the terminal commands you can execute on a UNIX/Linx server to convert any text data file to use only LF line endings.
Explanation
The above code uses builtin function dos2unix to convert target_file.txt to a unix friendly tmp.txt file. We then use the mv command to overwrite the original file and delete the temporary file.
