• Home
• Getting started
• Documentation
• Support
• Get involved
• Blog
• Twitter

Configuring security for a particular URL

Introduction

This chapter deals with the config file's urls variable exclusively. While other parts of the documentation are about the general syntax of the config file, this one is concerned with how one should use the urls variable to secure a particular URL.

As a quick recap, here's how a sample config file may look like:

# -*- coding: utf-8 -*-
 
# stdlib
import uuid
 
# Don't share it with anyone.
INSTANCE_SECRET = '2573116c48444b7ba90632e16e60ac5f'
 
# May be shared with the outside world.
INSTANCE_UNIQUE = uuid.uuid4().hex
 
# ##############################################################################
 
def default():
    return {
        'ssl': True,
        'ssl-cert': True,
        'ssl-cert-commonName':INSTANCE_SECRET,
        'host': 'http://' + INSTANCE_SECRET
    }
 
urls = [
    ('/*', default()),
]

What we'll be dealing with is what goes into the urls variable, which should be a list of two-element lists or tuples. The first element is a pattern of the URL to be secured, the second one is a dictionary of options that apply to the pattern.

From now on, most of the chapter's examples will show only the urls variable but you still need to remember that it's merely a part, albeit an important one, of the whole configuration file.

URL patterns

Let's discuss the following configuration:

def config1():
    return {
        'ssl': True,
        'ssl-cert': True,
        'ssl-cert-commonName':'My Client1',
        'host': 'http://somewhere.example.com'
    }
 
def config2():
    return {
        'ssl': True,
        'ssl-cert': True,
        'ssl-cert-commonName':'My Client2',
        'host': 'http://somewhere.example.com'
    }
 
urls = [
    ('/client/1/<service_name:unicode>', config1()),
    ('/client/2/<service_name:any("customer", "billing")>', config2()),
]

There are two configuration dictionaries which we'll leave out for a second but what's important now is the URL patterns. sec-wall uses Pesto for pattern matching and this particular patterns should read:

• '/client/1/\<service_name:unicode>' - URL must start with the '/client/1/' string which is then followed by any Unicode expression, excluding '/', the path separator. Example URLs that will be picked up by this pattern may include '/client/1/invoice' and '/client/1/customer-account' whereas '/client/1/customer/cases' would not match.
• '/client/2/\<service_name:any("customer", "billing")>' - URL must start with the '/client/1/' string and may be followed only by what's listed in the any expression. So the only matching URLs will be '/client/2/customer' and '/client/2/billing' but any other won't match the pattern.

You sure have noticed that expressions are named, it's service_name in the examples above, and the names come in handy when configuring URL-rewriting.

In run-time, patterns are checked in the order they're given in the urls variable and the first match wins. The other ones are ignored and there won't be even any information in the logs that they would've been used hadn't there been duplicate entries.

The expression may be one of the following: int, unicode, path and any. A related concept is a /* (catch-all) pattern.

int

Matches any integer. Assuming a pattern in the form of '/foo/\<my_int:int>/bar', it would be matched by a '/foo/123/bar' URL but not if it were '/foo/zxc/bar'.

unicode

Matches any Unicode string, not including '/', the URL-path separator. If the pattern is '/foo/\<my_unicode:unicode>/bar' then both '/foo/123/bar' and '/foo/zxc/bar' will match it yet '/foo/zxc/qwe/bar' won't because there's a '/' in there.

path

Matches any URL path. In other words, it's like unicode but accepts '/' as well. All of the following will match the '/foo/\<my_path:path>/bar' pattern - '/foo/123/bar', '/foo/zxc/bar' and '/foo/zxc/qwe/bar'.

any

Indicates that a part of a URL must be within a range of listed values. If there's a '/foo/\<my_any:any("baz", "zxc")>/bar' pattern then the only two matching URLs will be '/foo/baz/bar' and '/foo/zxc/bar'.

/* (catch-all) pattern

'/' is a special-cased pattern not directly related to Pesto that catches any URL that wasn't picked up by patterns coming prior to it. Remembering the patterns are being checked in the order they're listed in the urls variable, you'll want to make sure '/' always comes at the end. Because the default strategy for handling URLs should be as secure as possible, the pattern will be automatically injected in the list even if you don't specify it yourself, although the exact behaviour can be customized with the add_default_if_not_found variable.

Security options

Once a URL pattern has been choosen, it needs to be paired with one and more of the security options listed below. Some of the options are required and others become required when certain parent options have been choosen.

The table below lists all the options broken out by the group they fall into and briefly explains whether the option becomes required under certain conditions, if at all, and what the default value is, if there is any.

You are required to provide the host requests are destined to, zero or more headers enrichment-related options and pick one of the security schemes: SSL/TLS client certificates, HTTP Basic Auth, HTTP Digest Auth, WS-Security, Custom HTTP headers, or XPath.

As a quick example, let's say that we want to let the client in only if the custom HTTP X-My-Secret header's value is equal to 'sSekreTT', here's how the config will look like:

{
    'host': 'http://somewhere.example.com',
    'custom-http': True,
    'custom-http-X-My-Secret':'sSekreTT'
}

The table's right below and if you need usage examples, just click here.

General options

host

Host the client requests should be proxied over to, including scheme (http:// or https://) and an optional port number of the backend server but no URL path because the path - unless URL-rewriting is in effect - will be copied over as-is.

That is, by default, if sec-wall is running on http://10.151.17.179:15000, host is 'http://10.152.3.191:18090' and client is accessing the http://10.151.17.179:15000/my/service URL, sec-wall will forward it to http://10.152.3.191:18090/my/service, without ever touching the URL path.

ssl

If True, indicates that the sec-wall's URL must be accessed through HTTPS, plain HTTP connections are forbidden - regardless of whether the client credentials had been supplied and were correct.

ssl-wrap-only

(new in 1.1)

If True, indicates that the sec-wall instance is to wrap the client connection to proxy using SSL/TLS without performing any authentication steps of whatsoever. In other words, the client connection will be encrypted but no authentication will be performed and any auth configuration will be ignored for the given URL pattern.

add-auth-info

Whether the backend should receive a YAML-formatted information regarding the credentials used by the client for accessing sec-wall. Backend may use the information for further restricting the access depending on what's in the YAML document sec-wall sends it through. The data is sent in the HTTP X-sec-wall-auth-info header and what's being forwarded depends on the security scheme which was being used.

• SSL/TLS client certificates - X-sec-wall-auth-info is a dictionary of relative distinguished names read from the client certificate's 'subject' field, as understood by Python, such as:

  {commonName: MyClientApp,
      countryName: US,
      stateOrProvinceName: Alaska
      localityName: Cordova,
      organizationName: MyCompany,
      organizationalUnitName: MyUnit}
  

  
  
• HTTP Basic Auth - X-sec-wall-auth-info is a dictionary with one key only, 'basic-auth-username', which indicates what username the client application used for accessing sec-wall, such as:

  {basic-auth-username: MyUsername}
  

  
  
• HTTP Digest Auth - X-sec-wall-auth-info is a dictionary with elements that point to the username and realm used by the client application when accessing the proxy, such as:

  {digest-auth-username: MyUsername, digest-auth-realm: My secret realm}
  

  
  
• WS-Security - X-sec-wall-auth-info is a dictionary with one key only, 'wsse-pwd-username', which indicates what username the client application used for accessing sec-wall, such as:

  {wsse-pwd-username: MyWSSEUsername}
  

  
  
• Custom HTTP headers - X-sec-wall-auth-info is a dictionary of HTTP headers the client was expected to send to sec-wall, that's including both their names and values. Let's say the client was to send two headers, 'Accept-Encoding' equal to 'gzip,deflate' and 'User-Agent' equal to 'Mozilla/5.0', in that case X-sec-wall-auth-info will have the value of:

  {Accept-Encoding: 'gzip,deflate', User-Agent: Mozilla/5.0}
  

  
  
• XPath - X-sec-wall-auth-info is a list of XPath that must've been been matched against the client request, such as:

  ['//my-elem[text()="my-expected-value"]']
  

  
  

sign-auth-info

Whether the X-sec-wall-auth-info should be cryptographically signed off. If yes, the value will sent to a backend server in the X-sec-wall-auth-info-signed HTTP header. The exact formula is as follows:

delimeter = ':'
invocation_id = as documented in the logging docs
instance_secret = INSTANCE_SECRET
auth_info = X-sec-wall-auth-info
X-sec-wall-auth-info-signed = SHA256(invocation_id + delimeter + instance_secret + delimeter + auth_info)

Headers enrichment

from-client-ignore

List of client headers that should be ignored and never forwarded to the backend server. Note that the names are treated case-insensitively.

Sample:

{
    'from-client-ignore': ['User-Agent', 'Connection']
    # .. skip other options
}

to-backend-add

A dictionary of headers to be injected by sec-wall while proxying requests to the backend server.

Sample:

{
    'to-backend-add': {'X-MyCustomHeader1': 'Hey there', 'X-MyCustomHeader2': "How's going, mate?"}
    # .. skip other options
}

from-backend-ignore

List of HTTP headers sent by a backend server that should be ignored and never forwarded to the client application.

{
    'from-backend-ignore': ['Server', 'Set-Cookie']
    # .. skip other options
}

to-client-add

A dictionary of headers to be injected by sec-wall while returning the backend responses to a client application.

Sample:

{
    'to-client-add': {'X-Powered-By': 'Python', 'Pragma': 'no-cache'}
    # .. skip other options
}

SSL/TLS client certificates

ssl-cert

If True, the client application must use an SSL/TLS certificate when connecting to sec-wall. The certificate needs to be signed off by one of the recognized CAs, as configured through the ca_certs variable.

Sample:

{
    'ssl': True,
    'ssl-cert': True
    # .. skip other options
}

ssl-cert-*

In addition to stating that a client application must use a certificate, it's also possible to require the certificate's fields be equal to agreed values. For instance, if you'd like the certificate not only be signed off by a trusted CA but also to have a 'commonName' equal to "MyClient" and 'locality' equal to 'Paris', here's how the configuration should look like:

{
    'ssl': True,
    'ssl-cert': True
    'ssl-cert-commonName': 'MyClient',
    'ssl-cert-locality': 'Paris'
    # .. skip other options
}

That is, the pattern is 'ssl-cert-' + FIELD_NAME: FIELD_VALUE, where FIELD_NAME is one of the certificate fields and FIELD_VALUE is the value you expect it to have.

HTTP Basic Auth

basic-auth

If True, the client application is expected to use HTTP Basic access authentication when connecting to a sec-wall instance.

Sample:

{
    'basic-auth': True
    # .. skip other options
}

basic-auth-username

The username for the client application to use when basic-auth is in effect.

{
    'basic-auth-username': 'MyUser'
    # .. skip other options
}

basic-auth-password

The password for the client application to use when basic-auth is in effect. Note that it needs to be in plain text.

{
    'basic-auth-password': 's~sSee!!Kr99e_>\eT'
    # .. skip other options
}

basic-auth-realm

The realm to use when sending an HTTP 401 status to the client application.

{
    'basic-auth-realm': 'Secure area'
    # .. skip other options
}

HTTP Digest Auth

digest-auth

If True, the client application is expected to use HTTP Digest access authentication as understood by RFC 2069 when connecting to a sec-wall instance.

Sample:

{
    'digest-auth': True
    # .. skip other options
}

digest-auth-username

The username for the client application to use when digest-auth is in effect.

Sample:

{
    'digest-auth-username': 'MyUser'
    # .. skip other options
}

digest-auth-password

The password for the client application to use when digest-auth is in effect. Note that it needs to be in plain text.

Sample:

{
    'digest-auth-password': 'Se++3Kre#t'
    # .. skip other options
}

digest-auth-realm

The realm to use when sending an HTTP 401 status to the client application.

{
    'digest-auth-realm': 'Secure area'
    # .. skip other options
}

WS-Security

wsse-pwd

Dictates whether client applications are required to use the Web Services Security: Username Token Profile V1.0 (PDF) when connecting to the proxy.

wsse-pwd-username

The username for the client application to use when wsse-pwd is in effect.

wsse-pwd-password

A clear-text password to use when the password type is http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText.

Note that wsse-pwd-password will have precedence if both wsse-pwd-password and wsse-pwd-password-digest are given.

wsse-pwd-realm

The name of the realm the process of authentication takes place in.

wsse-pwd-password-digest

A digest of the password to use when the password type is http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest.

Note that wsse-pwd-password will have precedence if both wsse-pwd-password and wsse-pwd-password-digest are given.

wsse-pwd-reject-empty-nonce-creation

Whether requests not containing the /soapenv:Envelope/soapenv:Header/wsse:Security/wsse:UsernameToken/wsse:Nonce element should be rejected. It's recommended to set the value to True.

wsse-pwd-reject-stale-tokens

Whether requests containing stale tokens, tokens whose expiry limit has elapsed, should be rejected. It's recommended to set the value to True and the time after which the token is considered to have staled is given in wsse-pwd-reject-expiry-limit.

wsse-pwd-reject-expiry-limit

The option is in seconds and says how long after the WS-Security token has been created it will still be considered fresh enough, that is, the client request will be rejected if token_creation_time + wsse-pwd-reject-expiry-limit > current_time.

wsse-pwd-nonce-freshness-time

The value must be set but currently sec-wall ignores it. It's meant to be used in future sec-wall versions to aid with preventing replay attacks. It's recommended to have it be equal to wsse-pwd-reject-expiry-limit.

Custom HTTP headers

custom-http

Whether client applications should be let in depending on the occurence and values of configured HTTP headers.

{
    'custom-http': True
    # .. skip other options
}

custom-http-*

There must be at least one HTTP configured when custom-http is True, such as in the example below where there are two of them - User-Agent is expected to be MyBrowser and X-MyHeader must be MySecret for sec-wall to let the request in.

{
    'custom-http': True,
    'custom-http-User-Agent':'MyBrowser',
    'custom-http-X-MyHeader':'MySecret'
    # .. skip other options
}

That is, the pattern is 'custom-http-' + HEADER_NAME: HEADER_VALUE, where HEADER_NAME is one of the HTTP headers and HEADER_VALUE is the value it must have.

XPath

xpath

If True, whether the client request will be let in or not will depend on a set of XPath expressions, each of which must evaluate to True.

{
    'xpath': True
    # .. skip other options
}

xpath-*

List of XPath expressions, each of them must evaluate to True for sec-wall to consider the request be valid, like below:

from lxml import etree
 
{
    'xpath': True,
    'xpath-expr1': etree.XPath('//username/value/text() = "MyUsername"'),
    'xpath-expr2': etree.XPath('//secret/value/text() = "MySecret"')
    # .. skip other options
}

That is, the pattern is 'xpath-' + EXPR_NAME: EXPR_VALUE, where EXPR_NAME is ignored and EXPR_VALUE is the value it must have. Note that EXPR_NAME may be anything as long as it's unique for a given URL configuration. EXPR_VALUE on the other hand is passed on directly to lxml for evaluation against the incoming requests. It is also possible to XML namespaces with XPath, consult this example for more information

Examples

There's a whole chapter devoted to nothing but showing various examples of sec-wall's configuration files, head over there for more information.

Exercises