Posted In: Apache, Commons

How to download large files using Apache commons IO

Apache IOUtils is already taking care of large files. Comments from IOUtils

    /**
     * Copy bytes from an <code>InputStream</code> to an
     * <code>OutputStream</code>.
     * <p>
     * This method buffers the input internally, so there is no need to use a
     * <code>BufferedInputStream</code>.
     * <p>
     * Large streams (over 2GB) will return a bytes copied value of
     * <code>-1</code> after the copy has completed since the correct
     * number of bytes cannot be returned as an int. For large streams
     * use the <code>copyLarge(InputStream, OutputStream)</code> method.
     * 
     * @param input  the <code>InputStream</code> to read from
     * @param output  the <code>OutputStream</code> to write to
     * @return the number of bytes copied
     * @throws NullPointerException if the input or output is null
     * @throws IOException if an I/O error occurs
     * @throws ArithmeticException if the byte count is too large
     * @since Commons IO 1.1
     */
    public static int copy(InputStream input, OutputStream output) throws IOException {
        long count = copyLarge(input, output);
        if (count > Integer.MAX_VALUE) {
            return -1;
        }
        return (int) count;
    }

FileUtils.copyURLToFile

package com.example;

import java.io.File;
import java.net.URL;

import org.apache.commons.io.FileUtils;

public class DownloadLargeFile {
	public static void main(String[] args) {
		new DownloadLargeFile().proceed();
	}

	public void proceed() {
		System.out
		        .println("DownloadLargeFile::proceed() started");
		String path = "https://www.nseindia.com/content/historical/EQUITIES/2017/SEP/cm19SEP2017bhav.csv.zip";
		try {
			URL url = new URL(path);
			File initialFile = new File(
			        "E:/abhijit/cm19SEP2017bhav.zip");
			FileUtils.copyURLToFile(url, initialFile);
		} catch (

		Exception e) {
			e.printStackTrace();
		}
	}
}

Tags:

by , on September 20th, 2017

  • Categories