https://www.cise.ufl.edu/~msr/tag/pandas/ pandas Wowchemy (https://wowchemy.com)en-us© 2021 - Sajid RahmanThu, 07 Oct 2021 00:00:00 +0000 https://www.cise.ufl.edu/~msr/media/icon_hu257dded1a2315d693952aa7ac83b6d59_5868_512x512_fill_lanczos_center_3.png https://www.cise.ufl.edu/~msr/tag/pandas/ https://www.cise.ufl.edu/~msr/post/misc-troubleshooting/ Thu, 07 Oct 2021 00:00:00 +0000 https://www.cise.ufl.edu/~msr/post/misc-troubleshooting/ <h3 id="fixing-_csverror-field-larger-than-field-limit-131072">Fixing <code>_csv.Error: field larger than field limit (131072)</code></h3> <p><strong>Solution</strong>: We need to increase <code>csv.field_size_limit</code>. The following code snippet shows the configuration how to increase default csv field limit. <strong>Note</strong>, this technique is also useful if you run into this issue while workding with Panda&rsquo;s <code>pd.read_csv()</code> method as well.</p> <pre><code class="language-python">import csv #if you're not sure what'd be the maximum field size, use system maxsize field_size_limit = sys.maxsize while True: try: csv.field_size_limit(field_size_limit) break except OverflowError: field_size_limit = int(field_size_limit / 10) </code></pre>