htmlUnit介绍:
htmlunit 是一款开源的java 页面分析工具，读取页面后，可以有效的使用htmlunit分析页面上的内容。项目可以模拟浏览器运行，被誉为java浏览器的开源实现。是一个没有界面的浏览器，运行速度迅速。官方网址:
http://htmlunit.sourceforge.net/
htmlUnit示例:
pom依赖:
<dependency> <groupId>net.sourceforge.htmlunit</groupId> <artifactId>htmlunit</artifactId> <version>2.26</version> </dependency>
测试代码：
import com.gargoylesoftware.htmlunit.WebClient;import com.gargoylesoftware.htmlunit.html.HtmlInput;import com.gargoylesoftware.htmlunit.html.HtmlPage;import org.junit.Test;import java.io.IOException;public class BaiduDemo { @Test public void test() throws IOException { WebClient webClient = new WebClient(); webClient.getOptions().setJavaScriptEnabled(false); webClient.getOptions().setCssEnabled(false); webClient.getOptions().setUseInsecureSSL(false); //获取页面 String url =”https://www.baidu.com”; HtmlPage page = webClient.getPage(url); System.out.println(“页面文本:”+page.getTitleText()); //获取页面元素 HtmlInput htmlInput = page.getHtmlElementById(“kw”); System.out.println(htmlInput.asText()); htmlInput.setValueAttribute(“test”); HtmlInput btn = page.getHtmlElementById(“su”); HtmlPage page2 = btn.click(); System.out.println(“页面2:”+page2.getTitleText()); } }
获取cookie:
BasicCookieStore cookieStore = new BasicCookieStore(); Set<Cookie> cookies2 =webClient.getCookieManager().getCookies(); for (Cookie cookie : cookies2) { cookieStore一分快三计划导师QQ群号n(“页面2:”+page2.getTitleText()); } }
获取cookie:
BasicCookieStore cookieStore = new BasicCookieStore(); Set<Cookie> cookies2 =webClient.getCookieManager().getCookies(); for (Cookie cookie : cookies2) { cookieStore.addCookie(cookie.toHttpClient()); }
通过htmlUnit获取的cookie信息发送https请求:
入参:
cookieStore //通过htmlUnit获取的登陆信息
url参数:要访问的页面参数
protected String sendRequest(BasicCookieStore cookieStore,String url) throws Exception { if (null == cookieStore) { return null; } //设置cookie HttpClientContext context = HttpClientContext.create(); context.setCookieStore(cookieStore); //忽略SSL证书验证 SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() { //信任所有 public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException { return true; } }).build(); CloseableHttpClient httpClient = HttpClients.custom().setSSLContext(sslContext).build(); HttpGet httpGet = new HttpGet(url); //发送请求并获取response CloseableHttpResponse response = httpClient.execute(httpGet, context); HttpEntity entity = response.getEntity(); return EntityUtils.toString(entity); }
htmlUnit应用场景
todo
相关资料:
http://blog.csdn.net/lifj07/article/details/8638098