Options
All
  • Public
  • Public/Protected
  • All
Menu

settings related to requesting internet resources (your page and resources referenced by your page)

Hierarchy

  • IRequestSettings

Index

Properties

Optional authentication

authentication: { password: string; userName: string }

username/password for simple HTTP authentication

Type declaration

  • password: string
  • userName: string

Optional clearCache

clearCache: boolean

if true, clears cache between chained IPageRequest navigations. note: only important if multiple pages are navigated in one IUserRequest. cache is never shared between api calls.

default

false

Optional clearCookies

clearCookies: boolean

if true, clears all cookies upon initial navigation to the targetUrl. consider using the deleteCookies property for targeted removals. note: only important if multiple pages are navigated in one IUserRequest. cookies are never shared between api calls.

default

false

Optional cookies

cookies: ICookie[]

You must specify name, value, and either domain or url. Set Cookies for any domain, prior to loading this IPageRequest. If a cookie already exists with the same domain+path+name combination, it will be replaced. See ICookie for documentation on the cookie parameters.

Optional customHeaders

customHeaders: {}

BEWARE: setting custom headers can corrupt your request. use with care. specify additional request headers here. They will be sent to the server for every request issued (the page and resources). Unicode is not supported (ASCII only) example: customHeaders:{"myHeader":"myValue","yourHeader":"someValue"} if you want to set headers for just the target page (and not every sub-request) use the IUrlSettings.headers parameter.

Type declaration

  • [key: string]: string

Optional deleteCookies

deleteCookies: string[]

delete any cookie with a matching "name" property before processing the request.

Optional disableJavascript

disableJavascript: boolean

set to true to disable all Javascript from being processed on your page.

Optional disableSecureHeaders

disableSecureHeaders: boolean

new for Chrome backend (not supported in WebKit).

if set to true, rewrites your request to disable content-security-policy, xss protection, and CORS, assuming that webSecurityEnabled and xssAuditingEnabled settings do not conflict..

IMPORTANT This is an experimental feature, which is why it is FALSE by default. when it is more stable, it will be on by default and will need to be turned off via enableSecureHeaders

default

false

Optional doneWhen

doneWhen: Array<IDoneWhen>

new for Chrome backend. (not available on WebKit).

Render the page early, once a specific criteria is found on the page.

Syntax: doneWhen = Array of IDoneWhen objects.

Experimental: There are some bugs, and this future may be adjusted in the comming months based on usage feedback. If you have any comments/questions, please email Support@PhantomJsCloud.com

As soon as your criteria is met, will trigger the completion of your apge (But still respects waitInterval delay and in-progress injected scripts). If the all the criteria is not met, the page will continue waiting, until the maxWait is timeout reached.

You may pass an array so that you can detect modify the response status code, and whichever element matches will be noted in the userResponse.content.doneWhen node, and in the pjsc-content-done-when HTTP Response Header.

Usage Notes:

1) each doneWhen element may have one or more criteria. All criteria must be met for the load to be considered done. Pass multiple array elements if you want "OR" style functionality.

2) Scanning for your critera is performed aproximately every 50ms. This means that rapid changes to the web page may be overlooked.

3) If you need finer control of completion use IScriptPjscMeta.manualWait and/or IScriptPjscMeta.forceFinish by injecting a custom script via IPageRequest.scripts.

example
{
url:"https://PhantomJsCloud.com/examples/corpus/ajax.html",
"requestSettings":{
"doneWhen":[
{"text":'"statusCode":206',statusCode:202},
{"selector":"pre#fill-target",statusCode:201},
],
waitInterval:0,
},
renderSettings:{
passThroughStatusCode:true,
},
}

Optional emulateDevice

emulateDevice: "random" | "iPhone 4" | "iPhone 4 landscape" | "iPhone X" | "iPhone X landscape" | "Nexus 4" | "Nexus 4 landscape" | "Nexus 10" | "Nexus 10 landscape" | "Windows IE11 1080p" | "Windows IE11 1080p landscape" | "Surface 3 Chrome" | "Surface 3 Chrome landscape" | "googlebot"

new for Chrome backend (not supported in WebKit). if set, will override the viewport and useragent.

Optional ignoreImages

ignoreImages: boolean

set to true to skip loading of inlined images. If you are not outputing a screenshot, you can usually set this to true, which will decrease load times.

Optional ioWait

ioWait: number

new for Chrome backend. (not available on WebKit).

maximum amount of time (in ms) to wait when communicating with the browser, such as when retrieving iframes or cookies, detecting page height, etc. Default is 2000ms. You usually will not need to change this, unless dealing with JSON output and very large/complex pages.

Optional maxWait

maxWait: number

The maximum amount of time (ms timeout) you wish to wait for the target page to finish loading. Default is 35000 (35 seconds).

When rendering a page, we will give you whatever is ready at this time (page may be incompletely loaded).

Can be increased up to 5 minutes (300000) , but that only should be used as a last resort, as it is a relatively expensive page render (you are billed for render time).

if this value is exceeded, the current page will be rendered "normally" (status 200), however you may inspect the target page's status code by looking at pjsc-content-status-code header, or content.statusCode if outputting as JSON.

Optional recordResourceBody

recordResourceBody: string

new for Chrome backend. (not available on WebKit).

A regex. If it maches a resource's response URL, we will record the response body in base64 format, stored in the output JSON under the pageResponses.events (key=response) node.

For advanced use. This is useful when you want to inspect data sent to your page, but is otherwise not rendered in the page output.

example
{
"url": "https://www.example.com",
"renderType": "jpeg",
"outputAsJson": true,
"requestSettings": {
"recordResourceBody": "https\\://www.example.com.*" //if you need an escape character (slash) be sure to double-escape, as shown in this example
}
}

Optional resourceModifier

resourceModifier: IResourceModifier[]

array of regex + adjustment parametes for modifying or rejecting resources being loaded by the webpage.

example:
//screenshot of amazon.com using a random ip, without images, fonts, or css
{
url:"https://www.amazon.com",
renderType:"jpeg",
proxy:"anon-any",
requestSettings:{
resourceModifier:[
{type:"stylesheet" ,isBlacklisted:true},
{type:"font" ,isBlacklisted:true},
{type:"image" ,isBlacklisted:true},
],
},
}
example:
//load a page blacklisting css, and setting additional headers for all resources from mydomain.com
{ url:"http://www.example.com",
requestSettings:{
  "resourceModifier": [{regex:".*css.*",isBlacklisted:true}{"regex": "http://mydomain.com.*","setHeader": {"hello": "world","Accept-encoding": "tacos"}}]
},
}

Optional resourceTimeout

resourceTimeout: number

maximum amount of time to wait for each external resource to load. we kill the request if it exceeds this amount.

Optional resourceWait

resourceWait: number

maximum amount of time (in ms) to wait for each external resources to load. (.js, .png, etc) if the time exceeds this, we don't cancel the resource request, but we don't delay rendering the IPageRequest if everything else is done.

Optional stopOnError

stopOnError: boolean

if true, will stop IPageRequest load upon the first error detected, and move to next phase (render or next page)

Optional userAgent

userAgent: string

default useragent is "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/534.34 (KHTML, like Gecko) Safari/534.34 PhantomJS/2.0.0 (PhantomJsCloud.com/2.0.1)"

Optional waitInterval

waitInterval: number

Milliseconds to delay rendering after the last resource is finished loading (default is 1000ms). This is useful in case there are any AJAX requests or animations that need to finish up. If additional network requests are made while we are waiting, the waitInterval will restart once finished again. This can safely be set to 0 if you know there are no AJAX or animations you need to wait for (decreasing your billed costs)

Optional webSecurityEnabled

webSecurityEnabled: boolean

set to true to enable web security. This includes things like CORS and CSP (content security policy).

You may also set the experimental disableSecureHeaders property to further reduce page security.

IMPORTANT: only the first IPageRequest can set this property, and it is reused for the remainder of your request.

default

false

Optional xssAuditingEnabled

xssAuditingEnabled: boolean

set to true to prohibit cross-site scripting attempts (XSS).

You may also set the experimental disableSecureHeaders property to further reduce page security.

default

false