org.codelibs.elasticsearch.web.robot.transformer

クラス ScrapingTransformer

java.lang.Object

org.codelibs.robot.transformer.impl.AbstractTransformer

org.codelibs.robot.transformer.impl.HtmlTransformer

org.codelibs.elasticsearch.web.robot.transformer.ScrapingTransformer

すべての実装されたインタフェース:

org.codelibs.robot.transformer.Transformer

public class ScrapingTransformer
extends org.codelibs.robot.transformer.impl.HtmlTransformer

フィールドのサマリー

フィールド

修飾子とタイプ	フィールドと説明
protected ThreadLocal<Set<String>>	childUrlSetLocal
String[]	copiedResonseDataFields
protected RiverConfig	riverConfig

クラスから継承されたフィールド org.codelibs.robot.transformer.impl.HtmlTransformer

childUrlRuleMap, defaultEncoding, featureMap, invalidUrlPattern, LOCATION_HEADER, preloadSizeForCharset, propertyMap

クラスから継承されたフィールド org.codelibs.robot.transformer.impl.AbstractTransformer

name

コンストラクタのサマリー

コンストラクタ

コンストラクタと説明
ScrapingTransformer()

メソッドのサマリー

修飾子とタイプ	メソッドと説明
protected void	addPropertyData(Map<String,Object> dataMap, String key, Object value)
protected void	convertFlatMap(String prefix, Map<String,Object> oldMap, Map<String,Object> newMap)
protected void	deepCopy(Map<String,Object> oldMap, Map<String,Object> newMap)
Object	getData(org.codelibs.robot.entity.AccessResultData accessResultData) Returns data as XML content of String.
protected org.jsoup.nodes.Element[]	getElements(org.jsoup.nodes.Element[] elements, List<String> queries, boolean isArrayProperty)
protected org.jsoup.nodes.Element[]	getElements(org.jsoup.nodes.Element[] elements, String query)
protected Method	getQueryMethod(org.jsoup.nodes.Element element, String queryType, List<Object> argList)
protected org.codelibs.elasticsearch.web.robot.transformer.ScrapingTransformer.ScriptInfo	getScriptValue(Map<String,Object> params)
void	init()
protected String	loadCharset(InputStream inputStream, int preloadSize)
protected void	processCssQuery(org.jsoup.nodes.Document document, String propName, Map<String,Object> params, boolean isTrimSpaces, List<String> strList)
protected void	processData(ScrapingRule scrapingRule, File file, org.codelibs.robot.entity.ResponseData responseData, org.codelibs.robot.entity.ResultData resultData)
protected void	storeChildUrls(org.codelibs.robot.entity.ResponseData responseData, org.codelibs.robot.entity.ResultData resultData)
protected void	storeData(org.codelibs.robot.entity.ResponseData responseData, org.codelibs.robot.entity.ResultData resultData)
protected void	storeIndex(org.elasticsearch.client.Client client, String indexName, String typeName, Map<String,Object> dataMap)
protected void	storeIndex(org.codelibs.robot.entity.ResponseData responseData, Map<String,Object> dataMap)
org.codelibs.robot.entity.ResultData	transform(org.codelibs.robot.entity.ResponseData responseData)
protected String	trimSpaces(String value, boolean trimSpaces)
protected void	updateCharset(org.codelibs.robot.entity.ResponseData responseData)

クラスから継承されたメソッド org.codelibs.robot.transformer.impl.HtmlTransformer

addChildUrlFromTagAttribute, addChildUrlRule, addFeature, addProperty, convertChildUrlList, encodeUrl, getBaseHref, getChildUrlRuleMap, getDefaultEncoding, getDomParser, getDuplicateUrl, getFeatureMap, getInvalidUrlPattern, getPreloadSizeForCharset, getPropertyMap, getUrlFromTagAttribute, getXPathAPI, isHtml, isSupportedCharset, isValidPath, loadCharset, normalizeUrl, parseCharset, setChildUrlRuleMap, setDefaultEncoding, setFeatureMap, setInvalidUrlPattern, setPreloadSizeForCharset, setPropertyMap

クラスから継承されたメソッド org.codelibs.robot.transformer.impl.AbstractTransformer

getName, setName

クラスから継承されたメソッド java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

フィールドの詳細

copiedResonseDataFields

public String[] copiedResonseDataFields

riverConfig

protected RiverConfig riverConfig

childUrlSetLocal

protected ThreadLocal<Set<String>> childUrlSetLocal

コンストラクタの詳細

ScrapingTransformer

public ScrapingTransformer()

メソッドの詳細

init

public void init()

transform

public org.codelibs.robot.entity.ResultData transform(org.codelibs.robot.entity.ResponseData responseData)

定義:

transform インタフェース内 org.codelibs.robot.transformer.Transformer

オーバーライド:

transform クラス内 org.codelibs.robot.transformer.impl.HtmlTransformer

updateCharset

protected void updateCharset(org.codelibs.robot.entity.ResponseData responseData)

オーバーライド:

updateCharset クラス内 org.codelibs.robot.transformer.impl.HtmlTransformer

loadCharset

protected String loadCharset(InputStream inputStream,
                             int preloadSize)

storeData

protected void storeData(org.codelibs.robot.entity.ResponseData responseData,
                         org.codelibs.robot.entity.ResultData resultData)

オーバーライド:

storeData クラス内 org.codelibs.robot.transformer.impl.HtmlTransformer

processData

protected void processData(ScrapingRule scrapingRule,
                           File file,
                           org.codelibs.robot.entity.ResponseData responseData,
                           org.codelibs.robot.entity.ResultData resultData)

getScriptValue

protected org.codelibs.elasticsearch.web.robot.transformer.ScrapingTransformer.ScriptInfo getScriptValue(Map<String,Object> params)

processCssQuery

protected void processCssQuery(org.jsoup.nodes.Document document,
                               String propName,
                               Map<String,Object> params,
                               boolean isTrimSpaces,
                               List<String> strList)

getQueryMethod

protected Method getQueryMethod(org.jsoup.nodes.Element element,
                                String queryType,
                                List<Object> argList)

getElements

protected org.jsoup.nodes.Element[] getElements(org.jsoup.nodes.Element[] elements,
                                                List<String> queries,
                                                boolean isArrayProperty)

getElements

protected org.jsoup.nodes.Element[] getElements(org.jsoup.nodes.Element[] elements,
                                                String query)

trimSpaces

protected String trimSpaces(String value,
                            boolean trimSpaces)

addPropertyData

protected void addPropertyData(Map<String,Object> dataMap,
                               String key,
                               Object value)

storeIndex

protected void storeIndex(org.codelibs.robot.entity.ResponseData responseData,
                          Map<String,Object> dataMap)

storeIndex

protected void storeIndex(org.elasticsearch.client.Client client,
                          String indexName,
                          String typeName,
                          Map<String,Object> dataMap)

deepCopy

protected void deepCopy(Map<String,Object> oldMap,
                        Map<String,Object> newMap)

convertFlatMap

protected void convertFlatMap(String prefix,
                              Map<String,Object> oldMap,
                              Map<String,Object> newMap)

storeChildUrls

protected void storeChildUrls(org.codelibs.robot.entity.ResponseData responseData,
                              org.codelibs.robot.entity.ResultData resultData)

オーバーライド:

storeChildUrls クラス内 org.codelibs.robot.transformer.impl.HtmlTransformer

getData

public Object getData(org.codelibs.robot.entity.AccessResultData accessResultData)

Returns data as XML content of String.

定義:

getData インタフェース内 org.codelibs.robot.transformer.Transformer

オーバーライド:

getData クラス内 org.codelibs.robot.transformer.impl.HtmlTransformer

戻り値:

XML content of String.