CrawlerTransformConfig

Transform a input column of urls to html text

KEY TYPE Description
outputColumn str Name of output column to store transformed data.
crawlDepth None None
inputColumnType None None
disableHostRestriction bool If True, will not restrict crawling to the same host.
userAgent str If provided, will use this user agent instead of randomly selecting one.
depthColumn str Increasing depth explores more links, capturing more content
honourWebsiteRules bool If True, will respect robots.txt rules.
inputColumn str Name of input column to transform.
© Copyright