Skip to content

Commit

Permalink
Merge pull request #33 from logandk/requirements-in-root
Browse files Browse the repository at this point in the history 
Improve requirement packaging
  • Loading branch information
@logandk
logandk authored Sep 29, 2017
2 parents 15fa129 + 2fdcbcd commit dce22f8
Show file tree
Hide file tree
Showing 7 changed files with 461 additions and 223 deletions.
8 changes: 8 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
# 1.4.0
## Features
* Package requirements into service root directory in order to avoid munging
sys.path to load requirements (#30).
* Package requirements when deploying individual non-WSGI functions (#30).
* Added `pythonBin` option to set python executable, defaulting to current runtime version (#29).


# 1.3.1
## Features
* Add configuration for handling base path mappings (API_GATEWAY_BASE_PATH)
Expand Down
30 changes: 18 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

A Serverless v1.x plugin to build your deploy Python WSGI applications using Serverless. Compatible
WSGI application frameworks include Flask, Django and Pyramid - for a complete list, see:
[http://wsgi.readthedocs.io/en/latest/frameworks.html](http://wsgi.readthedocs.io/en/latest/frameworks.html).
http://wsgi.readthedocs.io/en/latest/frameworks.html.

### Features

Expand Down Expand Up @@ -130,7 +130,7 @@ Serverless: Stack update finished...
Set `custom.wsgi.app` in `serverless.yml` according to your WSGI callable:

* For Pyramid, use [make_wsgi_app](http://docs.pylonsproject.org/projects/pyramid/en/latest/api/config.html#pyramid.config.Configurator.make_wsgi_app) to intialize the callable
* Django is configured for WSGI by default, set the callable to `<project_name>.wsgi.application`. See [https://docs.djangoproject.com/en/1.10/howto/deployment/wsgi/](https://docs.djangoproject.com/en/1.10/howto/deployment/wsgi/) for more information.
* Django is configured for WSGI by default, set the callable to `<project_name>.wsgi.application`. See https://docs.djangoproject.com/en/1.10/howto/deployment/wsgi/ for more information.


## Usage
Expand All @@ -147,30 +147,36 @@ Flask==0.12.2
requests==2.18.3
```

For more information, see [https://pip.readthedocs.io/en/1.1/requirements.html](https://pip.readthedocs.io/en/1.1/requirements.html).
For more information, see https://pip.readthedocs.io/en/1.1/requirements.html.

You can use the requirement packaging functionality of *serverless-wsgi* without the WSGI
handler itself by including the plugin in your `serverless.yml` configuration, without specifying
the `custom.wsgi.app` setting. This will omit the WSGI handler from the package, but include
any requirements specified in `requirements.txt`.

If you do not include the WSGI handler, you'll need to add `.requirements` to the Python search path
manually in your handler, before importing any packages:
If you don't want to use automatic requirement packaging you can set `custom.wsgi.packRequirements` to false:

```
import os
import sys
root = os.path.abspath(os.path.join(os.path.dirname(__file__)))
sys.path.insert(0, os.path.join(root, '.requirements'))
```yaml
custom:
wsgi:
app: api.app
packRequirements: false
```
If you don't want to use automatic requirement packaging you can set `custom.wsgi.packRequirements` to false:
For a more advanced approach to packaging requirements, consider using https://github.com/UnitedIncome/serverless-python-requirements.
### Python version
Python is used for packaging requirements and serving the app when invoking `sls wsgi serve`. By
default, the current runtime setting is expected to be the name of the Python binary in `PATH`,
for instance `python3.6`. If this is not the name of your Python binary, override it using the
`pythonBin` option:

```yaml
custom:
wsgi:
app: api.app
packRequirements: false
pythonBin: python3
```

### Local server
Expand Down
108 changes: 93 additions & 15 deletions index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,11 @@ BbPromise.promisifyAll(fse);
class ServerlessWSGI {
validate() {
this.enableRequirements = true;
this.pythonBin = this.serverless.service.provider.runtime;

if (this.serverless.service.custom && this.serverless.service.custom.wsgi) {
this.pythonBin = this.serverless.service.custom.wsgi.pythonBin || this.pythonBin;

if (this.serverless.service.custom.wsgi.app) {
this.wsgiApp = this.serverless.service.custom.wsgi.app;
this.appPath = path.dirname(path.join(this.serverless.config.servicePath, this.wsgiApp));
Expand All @@ -25,14 +28,18 @@ class ServerlessWSGI {

this.serverless.service.package = this.serverless.service.package || {};
this.serverless.service.package.include = this.serverless.service.package.include || [];
this.serverless.service.package.exclude = this.serverless.service.package.exclude || [];

var includes = ['wsgi.py', '.wsgi_app'];
this.serverless.service.package.include = _.union(
this.serverless.service.package.include,
['wsgi.py', '.wsgi_app']);

if (this.enableRequirements) {
includes.push('.requirements/**');
this.requirementsInstallPath = path.join(
this.appPath ? this.appPath : this.serverless.config.servicePath,
'.requirements');
this.serverless.service.package.exclude.push('.requirements/**');
}

this.serverless.service.package.include = _.union(this.serverless.service.package.include, includes);
}

packWsgiHandler() {
Expand All @@ -56,7 +63,6 @@ class ServerlessWSGI {
packRequirements() {
const requirementsPath = this.appPath || this.serverless.config.servicePath;
const requirementsFile = path.join(requirementsPath, 'requirements.txt');
const requirementsInstallPath = this.appPath ? this.appPath : this.serverless.config.servicePath;
let args = [path.resolve(__dirname, 'requirements.py')];

if (!this.enableRequirements) {
Expand All @@ -75,12 +81,12 @@ class ServerlessWSGI {
}
}

args.push(path.join(requirementsInstallPath, '.requirements'));
args.push(this.requirementsInstallPath);

this.serverless.cli.log('Packaging required Python packages...');

return new BbPromise((resolve, reject) => {
const res = child_process.spawnSync('python', args);
const res = child_process.spawnSync(this.pythonBin, args);
if (res.error) {
return reject(res.error);
}
Expand All @@ -91,12 +97,63 @@ class ServerlessWSGI {
});
}

cleanup() {
const artifacts = ['wsgi.py', '.wsgi_app'];
linkRequirements() {
if (!this.enableRequirements) {
return BbPromise.resolve();
}

if (this.enableRequirements) {
artifacts.push('.requirements');
if (fse.existsSync(this.requirementsInstallPath)) {
this.serverless.cli.log('Linking required Python packages...');

fse.readdirSync(this.requirementsInstallPath).map((file) => {
this.serverless.service.package.include.push(file);
this.serverless.service.package.include.push(`${file}/**`);

try {
fse.symlinkSync(`${this.requirementsInstallPath}/${file}`, file);
} catch (exception) {
let linkConflict = false;
try {
linkConflict = (fse.readlinkSync(file) !== `${this.requirementsInstallPath}/${file}`);
} catch (e) {
linkConflict = true;
}
if (linkConflict) {
throw new this.serverless.classes.Error(
`Unable to link dependency '${file}' ` +
'because a file by the same name exists in this service');
}
}
});
}
}

unlinkRequirements() {
if (!this.enableRequirements) {
return BbPromise.resolve();
}

if (fse.existsSync(this.requirementsInstallPath)) {
this.serverless.cli.log('Unlinking required Python packages...');

fse.readdirSync(this.requirementsInstallPath).map((file) => {
if (fse.existsSync(file)) {
fse.unlinkSync(file);
}
});
}
}

cleanRequirements() {
if (!this.enableRequirements) {
return BbPromise.resolve();
}

return fse.removeAsync(this.requirementsInstallPath);
}

cleanup() {
const artifacts = ['wsgi.py', '.wsgi_app'];

return BbPromise.all(_.map(artifacts, (artifact) =>
fse.removeAsync(path.join(this.serverless.config.servicePath, artifact))));
Expand All @@ -106,7 +163,7 @@ class ServerlessWSGI {
const providerEnvVars = this.serverless.service.provider.environment || {};
_.merge(process.env, providerEnvVars);

_.each(this.serverless.service.functions, function (func) {
_.each(this.serverless.service.functions, (func) => {
if (func.handler == 'wsgi.handler') {
const functionEnvVars = func.environment || {};
_.merge(process.env, functionEnvVars);
Expand All @@ -125,7 +182,7 @@ class ServerlessWSGI {
const port = this.options.port || 5000;

return new BbPromise((resolve, reject) => {
var status = child_process.spawnSync('python', [
var status = child_process.spawnSync(this.pythonBin, [
path.resolve(__dirname, 'serve.py'),
this.serverless.config.servicePath,
this.wsgiApp,
Expand Down Expand Up @@ -158,6 +215,12 @@ class ServerlessWSGI {
},
},
},
clean: {
usage: 'Remove cached requirements.',
lifecycleEvents: [
'clean',
],
},
},
},
};
Expand All @@ -166,29 +229,44 @@ class ServerlessWSGI {
'before:deploy:createDeploymentArtifacts': () => BbPromise.bind(this)
.then(this.validate)
.then(this.packWsgiHandler)
.then(this.packRequirements),
.then(this.packRequirements)
.then(this.linkRequirements),

'after:deploy:createDeploymentArtifacts': () => BbPromise.bind(this)
.then(this.validate)
.then(this.unlinkRequirements)
.then(this.cleanup),

'wsgi:serve:serve': () => BbPromise.bind(this)
.then(this.validate)
.then(this.loadEnvVars)
.then(this.serve),

'wsgi:clean:clean': () => BbPromise.bind(this)
.then(this.validate)
.then(this.unlinkRequirements)
.then(this.cleanRequirements)
.then(this.cleanup),

'before:deploy:function:packageFunction': () => BbPromise.bind(this)
.then(() => {
if (this.options.functionObj.handler == 'wsgi.handler') {
return BbPromise.bind(this)
.then(this.validate)
.then(this.packWsgiHandler)
.then(this.packRequirements);
.then(this.packRequirements)
.then(this.linkRequirements);
} else {
return BbPromise.bind(this)
.then(this.validate)
.then(this.packRequirements)
.then(this.linkRequirements);
}
}),

'after:deploy:function:packageFunction': () => BbPromise.bind(this)
.then(this.validate)
.then(this.unlinkRequirements)
.then(this.cleanup)
};
}
Expand Down
Loading

0 comments on commit dce22f8

Please sign in to comment.