# How to read files ... one line at a time

##### Posted on June 01, 2012

Imagine that we have a file (.txt, .csv, .dat, etc) and that we need to do some kind of process for each line, and then save the results in another file. The only problem is that the size of the file is so huge that it is not possible to import it directly to R.

What can we do? We can simply read line by a line while applying the required operations and export the output to another file. Here’s how. We need the input file (where the data is), and the output file (where the results will be stored). The general idea is to:

1. Read the input file line by line
2. For each line:
1. Apply operations
2. Export the outcome to the output file
3. Repeat steps 2.1 and 2.2

### Toy example

Let’s say we have an input_file with a lot of lines. We need to apply some function to each line, and then export the outcome to an output_file. Let’s suppose that the outcome will be stored in a vector of length 6 with the following elements: Id id number, Name some name, Var1-Var3 are values for three variables, Status some label. Here’s one way to do it.

Published in categories how-to  Tagged with file  read  lines