博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
httpClient download file(爬虫)
阅读量:6416 次
发布时间:2019-06-23

本文共 1867 字,大约阅读时间需要 6 分钟。

package com.opensource.httpclient.bfs;

import java.io.DataOutputStream;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;

import org.apache.commons.httpclient.HttpStatus;

import org.apache.http.Header;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;

public class DownLoadFile

{
   
    public String getFileNameByUrl(String url, String contentType)
    {
        url = url.substring(7);
       
        if (contentType.indexOf("html") != -1)
        {
            url = url.replaceAll("[\\?/:*|<>\"]", "_") + ".html";
            return url;
        }
        else
        {
            return url.replaceAll("[\\?/:*|<>\"]", "_") + "." + contentType.substring(contentType.lastIndexOf("/") + 1);
        }
    }
   
    public void saveToLocal(byte[] data, String filePath)
    {
        try
        {
            DataOutputStream out = new DataOutputStream(new FileOutputStream(new File(filePath)));
            for (int i = 0; i < data.length; i++)
                out.write(data[i]);
            out.flush();
            out.close();
        }
        catch (IOException e)
        {
            e.printStackTrace();
        }
    }
   
    public String downloadFile(String url)
        throws ClientProtocolException, IOException
    {
        String filePath = null;
       
        HttpClient httpClient = new DefaultHttpClient();
       
        HttpGet get = new HttpGet(url);
       
        HttpResponse rsp = httpClient.execute(get);
       
        if (rsp.getStatusLine().getStatusCode() != HttpStatus.SC_OK)
        {
            System.err.println("Method failed: " + rsp.getStatusLine());
            filePath = null;
        }
        Header[] header = rsp.getHeaders("Content-Type");
        filePath = "D:\\" + getFileNameByUrl(url, header[0].getValue());
       
        saveToLocal(rsp.toString().getBytes(), filePath);
       
        return filePath;
    }
   
    public static void main(String[] args)
        throws ClientProtocolException, IOException
    {
        DownLoadFile downLoadFile = new DownLoadFile();
       
        String temp = downLoadFile.downloadFile("");
       
        System.out.println(temp);
    }
   
}

 

转载地址:http://lfpra.baihongyu.com/

你可能感兴趣的文章
choose MariaDB 10 or 5.x
查看>>
oVirt JBAS server start failed, ajp proxy cann't server correct. ovirt-engine URL cann't open
查看>>
CDP WebConsole上线公告
查看>>
ubuntu下安装摄像头应用程序xawtv
查看>>
GFS2,GFS,EXT2,EXT3 SEQ-write performance compare
查看>>
PostgreSQL 如何比较两个表的定义是否一致
查看>>
PHPCMS2008利用EXP
查看>>
Azure 安装.NET3.5机能错误一例
查看>>
扩展android-volley来开发Android restful client
查看>>
邻居子系统
查看>>
【转】一个程序员眼中的价值
查看>>
Python删除指定目录下的过期文件
查看>>
Linux Mint下Kindle Fire调试android程序
查看>>
自定义Background
查看>>
git笔记
查看>>
解决brew-cask 没有更新app的命令
查看>>
Android开源中国客户端学习 配置文件读写 以及其他一些工具类 <13>
查看>>
Android Browser学习十 快捷菜单模块: PieMenu的实现(2)
查看>>
国外的opencv识别文档
查看>>
Windows不能用鼠标双击运行jar文件怎么
查看>>