Miscellaneous Troubleshooting Tricks

Fixing _csv.Error: field larger than field limit (131072)

Solution: We need to increase csv.field_size_limit. The following code snippet shows the configuration how to increase default csv field limit. Note, this technique is also useful if you run into this issue while workding with Panda’s pd.read_csv() method as well.

import csv
#if you're not sure what'd be the maximum field size, use system maxsize
field_size_limit = sys.maxsize

while True:
    try:
        csv.field_size_limit(field_size_limit)
        break
    except OverflowError:
        field_size_limit = int(field_size_limit / 10)