add index plugin to list of community plugins

[jonschlinkert]

related to assemble/assemble#676

[vwochnik]

In the referring issue, are you asking me to port my plugin to assemble v0.6.0 and if so which changes do have to be made?

[jonschlinkert]

are you asking me to port my plugin to assemble v0.6.0

no, this would be a different project, we've been talking about publishing this as a combination of helpers, middleware and plugin(s), since pagination and indexing is such a big part of our plans for assemble.

which changes do have to be made?

@doowb and I started working on some pagination/index stuff a couple of weeks ago, the changes would actually be pretty extensive, aside from the pagination logic itself. e.g. the core concepts would be the same: get a template collection, loop over it, build up a pagination context, etc.

From a technical standpoint, if everything were documented, IMHO, the v0.6.0 API is much easier to use than what you had to do to get this working with assemble as a grunt plugin. But... we don't have docs yet so it's understandable if this is too much for someone to take on before the docs are done.

That said, if you or someone else wants to give this a shot, we'd love to see it as a core project, and we'd be happy to provide guidance as needed.

[vwochnik]

So, from my understanding:

• Assemble v0.6 is capabile of running standalone, thus all plugins need be only with assemble as well; no grunt whatsoever
• I've looked into the repository and installed the new assemble v0.6 globally (for cli -> need be in assemble-cli to not clutter up system) and assemble locally (/bin shouldn't be in there).
• Now, from my understanding I can create an assemblerc.yml which contains the configuration, plugins, middleware, etc.

Then, how do you configure individual tasks? Are the .pipe()s only available with gulp or how does the assemblerc.yml play together with the javascript stuff?

I'd really love to give this a try. Is there a repo with an example of the new version being used?

[jonschlinkert]

no grunt whatsoever

yes, that's correct

(for cli -> need be in assemble-cli to not clutter up system) and assemble locally (/bin shouldn't be in there).

that's the plan, we'll be moving the cli code to assemble-cli before release

I can create an assemblerc.yml which contains the configuration, plugins, middleware, etc.

Yes. We talked about auto-loading this, but for now you can load it using assemble.data('.assemble.yml'). Or, if you want to use that file for configuration you would need to read it in (and parse it as yaml) and pass it to assemble.option()

how do you configure individual tasks? Are the .pipe()s only available with gulp or how does the assemblerc.yml play together with the javascript stuff?

let me put an example together, that would be more useful for you I think

[vwochnik]

I'd appreciate this. Thanks.

[vwochnik]

BTW: I did a showt require(assemble) and started inspecting the objects. Why are there spaces inside object property names such as 'default engines'?

[jonschlinkert]

Why are there spaces inside object property names such as 'default engines'?

that comes from express, which was part of the inspiration for v0.6.0. It's only used on options that are defined like this:

assemble.enable('default engines');
assemble.disable('default engines');

Honestly, we went back and forth about that, but ultimately we decided it's fine (especially for built-in options) since it's far less likely to collide with any user-define values.

Here is the gist I started https://gist.github.com/jonschlinkert/e2da295ec7ca5d159914. Hope this helps. feel free to ask questions and I'll try to fill in blanks.

Probably the most exciting part of v0.6.0 (to me) is loaders, which aren't on the gist yet lol. I'll try to add them in a bit b/c I think they might come in handy with this.

[vwochnik]

Hi! I've successfully been able to run assemble with your provided examples but I have trouble specifying default layouts. In particular, how to use layoutdir, layoutext? Is it even necessary? I'd like to be able to specify my default layout in the assemble task and then I'd like to also be able to put it in YFM of individual pages.

Here's what I got so far:
https://github.com/vwochnik/assemble-v060-test

Layouts are not being used (since they are not specified) and the file extension is not changed (.md).

[jonschlinkert]

that's great!

how to use layoutdir, layoutext

These aren't needed.

default layout in the assemble task

Do:

// project-level default layout
assemble.option('layout', 'default');

// task-level default layout
assemble.task('html', function() {
  assemble.src('templates/*.hbs', {layout: 'default'})
    .pipe(assemble.dest('_gh_pages'));
});

file extension is not changed (.md).

I might need to whip up a markdown engine :) currently we only use helpers for markdown, but if we're going to do this index page thing, I think it might call for a proper markdown engine.

[vwochnik]

Alright, I got that step done. How am I now including the body within a given layout?

{{#markdown}}
  {{> body}}
{{/markdown}} 

isn't cutting it. I get the error Partial body not found..

Further, why not just use marked? Basically, for markdown all you need is

• parse front matter (not part of engine)
• (optionally) make a tree of headings for toc
• parse markdown to include into layouts

Or am I missing something?

[jonschlinkert]

You'll need to use {% body %}, sorry about that. One sec I'll link to another gist

[jonschlinkert]

actually by now you already have all of this done or figured out, but here it is in case it helps

[jonschlinkert]

You should still be able to do {{> foo }}, but also try using {{partial "foo"}}. either should work, but the second is a helper and will ultimately be more flexible

[vwochnik]

I got everything working that I had in the old version (ofc not the indices :) ) but how do I change file extension from .md to .html? I have tried setting options.ext to .html but it doesn't seem to work. Is there a .pipe() to change file extensions? This would be really awesome because then it would be possible to change the entire target directory structure as per description of a javascript function.

[jonschlinkert]

just drop in the extname plugin, like this:

var extname = require('gulp-extname');

assemble.task('html', function () {
  assemble.src('templates/*.hbs')
    .pipe(extname())
    .pipe(assemble.dest('dist/'));
});

it figures out the right extension to use based on the typeof source file type. it can be overridden but it covers a lot of common formats. we did it this way since assemble can work with any file type, not just html

[vwochnik]

So, I can say assemble.use(myModule) and myModule can then hook to various middleware stages? Also myModule can then add helpers? That would be really great, 'cause then I could do in a task

assemble.use(indexPagesModule).src("**/*.hbs", {/* assemble + middleware opts */})
    .pipe(...)
    .assemble.dest("dist/");

is this correct? This would make it really straight forward, even for beginners who barely know to use assemble because the module itself configures itself as a middleware and adds necessary helpers.

Could you then please give me an example using .use() where all middleware hooks are shown? You can also point me to the location of the source file defining these functions, I haven't found it in the assemble repository.

EDIT: I saw that .use() modules are called for EVERY template. IMO, that's not good because when I do index pages I need a hook .postRender() every template and then one .postRun() or .postTask() but I want .use() only to be executed once for the whole operation. Besides, when I hear the word use then I intuitively think include this.

Let's assume the code-snippet above would work, the module could look like this:

// this function is called by the .assembleuse() function with assemble being passed down as an argument.
module.exports = function(assemble) {
    // here, we can make hooks, create a collection of contexts, etc.
    var indexedItems = [];

    assemble.postRender(/.*/, function(context, next) {
        indexedItems.push(context);
    });

    assemble.postRun(function() {
        // build index pages by creating a new assemble command chain
    });
};

Then,

• the module could be used in multiple tasks and doesn't have to be rerequire()d
• modules can decide what to hook on and when
• it's easier for end users and plugin developers

Maybe we can rename the current use() into something else and use .use() for this purpose.

[doowb]

That's close, but the .use method is for running at all middleware methods or passing in a router to run on the specified middleware methods on the router.

the middleware methods we have right now are:

• onLoad => runs after templates are added to the cache.
• preRender => just before the template is rendered
• postRender => just after the template is rendered.

We have another method called .transform() which does what you're thinking of for .use: assemble.transform('myModule', myModule);

This was created to transform or add data to the assemble object, and it can also be used for adding middleware or creating custom templates types or whatever.... it's passed the instance of assemble as the first argument so you can use it like in your example.

Also, the middleware methods are only run on a "per-template" basis (this might be in a batch when going through assemble, but each one is checked for each template).

We would recommend making smaller middleware functions that collect information for the collections and export those functions so they can be assigned to a middleware based on an extension:

// index-items-middleware
module.exports = function (assemble) {
  var indexedItems = assemble.get('indexItems');
  return function (file, next) {
    indextedItems.push(file);
  };
};

// add the middleware
assemble.postRender(/\.md/, require('index-items-middleware')(assemble));

Then you can create a plugin that runs after the assemble.dest in the .pipe methods to use the indexed items:

var through = require('through2');
module.exports = function (assemble) {
  return through.obj(function (file, enc, cb) {
    // don't push the previous files into the stream
    cb();
  }, function (cb) {
    var stream = this;
    // create the index files and push them into the stream
    var indexes = createIndexFiles(assemble.get('indexItems'));
    indexes.forEach(function (index) {
      stream.push(index);
    });
    cb():
  });
};

Now you can use that plugin in your pipeline:

assemble.postRender(/\.md/, require('index-items-middleware')(assemble));
assemble.task('site', function () {
  return assemble.src('templates/posts/**/*.md')
    .pipe(extname())
    .pipe(assemble.dest('dist'))
    .pipe(require('index-plugin')(assemble))
    .pipe(assemble.dest('blog'));
});

I hope this helps or at least gets you closer. If there's something that needs explained in more detail, let me know.

[vwochnik]

Following questions:

• Can the .transform() method be used as per-task basis or will the loaded module persist in the assemble variable during the execution of another task?
• From a logical standpoint of view, it makes sense to divide the indexing/pagination into smaller pieces so that the user can easily specify what they want, I agree this is good. I have to think about how to put this into one package that the user can then require() once and use various parts of that package.
• The code above doesn't make sense to me. It should read within the task function assemble.postRender(...).src(...)...; because I only want the hook in that particular task. Does this make sense?
• By using assemble.dest() multiple times, the cache will be emptied each time?

How's this?

var assemble = require('assemble');
var Index = require('asemble-index-builder');

assemble.task('site', function () {
  var index = new Index({/*...*/});
  return assemble.postRender(/\.md/, index.middleware).src('templates/posts/**/*.md')
    .pipe(extname())
    .pipe(assemble.dest('dist'))
    .pipe(index())
    .pipe(assemble.dest('blog'));
});

This usage I find very intuitive.

[jonschlinkert]

Can the .transform() method be used as per-task basis

no, transforms are not task-specific.

This might help, Assemble inherits a number of these methods from Template. The tests in template are extensive, they might be useful to look at too.

assemble.postRender(...).src(...)...;

The middleware methods can't be chained with task methods. ... Actually, I just remembered that I wrote an overview that might help more with getting to know the API. Here is the intro to v0.6.0.

If possible I'll try to write more up today and post it when I have something to read.

[vwochnik]

I think I figured the best way to do modular plugins:

function doStuff() {

}

module.exports = function(app) {
  app.transform('template-index', function(app) {
    _.extend(app, {doStuff: doStuff});
  });
};

That way, I can extend the Template object (in this case Assemble) and I can create a method for instance app.extractDataFromTemplates(/*data property name*/, {/*options*/}) and another module with app.generateIndex('src/index.hbs', /*data property*/, {/*options*/}).

In your opinion is this a good way to do this? After all it interferes with the Templates and Assembles namespaces.

Furthermore, I understand that you do have access to the template contexts in the middleware but do you somehow also have access to those contexts by using the stream pipeline? In my through2.obj flush function, I do have access to the assemble object instance (and session) variables.

assemble.src(...)
.pipe(assemble.storeTemplateContexts(...))
.pipe(assemble.dest(...));

[jonschlinkert]

I wouldn't recommend that approach. all the methods needed to do this are all already available, so we should be able to accomplish this without creating new methods.

For example, you could build up a collection of tags from front-matter in a plugin or a .preRender() middleware, then in the flush function of a plugin, aggregate the tags and generate a list from a template.

here is a basic example: https://gist.github.com/jonschlinkert/debab424de26a0225cea
(edit: I moved the code example to a gist)

[jonschlinkert]

I just published an example project too: https://github.com/assemble/assemble-tags-collection-example

[vwochnik]

Wow! THIS is exactly what I needed. I didn't know file.data was in the plugin as well.

[jonschlinkert]

awesome, hope it helps!

[vwochnik]

I saw the plugin in lib/plugins and naturally I tried the following which does not work. The gist you gave had the plugin in the assemblefile.js which already has an assemble global variable but external plugins do not.

// plugin function called by `.pipe()`
module.exports = function() {
    var assemble = this; // this does not work

    return through2.obj(...);
}

It seems within the .task() function I do have access to this -> assemble but it is not passed down the stack. For now, I'm going with require('assemble-plugin-index')(assemble)

[jonschlinkert]

try doing it like this:

// my-plugin.js
module.exports = function(assemble) {
  return function (options) {
    return through.obj(function(file, enc, cb) {
      console.log(assemble.views)
      // do stuff to file
      this.push(file);
      cb();
    });
  };
}


// in the assemblefile.js
var plugin = require('./my-plugin.js')(assemble);

The paths plugin in lib/plugins uses .call() where the thisArg is the assemble object, which changes the invocation context to the assemble object. In other words, .call(assemble, foo, bar) makes this the assemble object.

[vwochnik]

As said, I'm interested in providing a plugin, so here it is:
https://github.com/vwochnik/assemble-plugin-index

I've moved the old code to v0.4 branch and created something new there. Please take a look at lib/index.js and tell me something about the code. I've put some TODOs in there where I don't know further.

To come

• Pagination
• File name generation pattern, e.g. index%% where %% is page
• Sorting of items, maybe with function for flexibility

[jonschlinkert]

it looks great! I'll do a pr in a minute

[vwochnik]

Thanks! I got your pull request! I hope you don't mind me asking a few questions

First a few questions regarding the changes you made

Thanks for the heads-up!

    return through2.obj(function(file, enc, cb) {
      // push the whole file through, so we can use any of its properties
      files.push(file);
      // through2 convention is to `push` the file (above), instead of passing
      // it to the callback
      cb();

So no this.push(file) is required? Because files is ours {} and not the stream. Also, cb(null,file);s straight from the docs! I'd keep it that way since it's the most performant and easy to read.

// actually load the template(s) defined by the user. this can be
    // a file path, or a glob pattern. the `indices()` method was created
    // above for this purpose.
    assemble.indices(glob);

But I've specified in the user docs that the user has to do it himself just like the layouts:

assemble.indices('templates/indices/*.hbs');

Or does assemble do it differently? You also say {layout: 'page'} in the options.

Also, I thought

// -- see above -- assemble.indices('templates/indices/*.hbs');

//...
.pipe(index('posts', {itemsPerPage: 10}))

Furthermore, is it possible if the user put this .pipe() before .assemble.dest that the templates passed through are rendered by the assemble core instead of manually by doing assemble.render()?

Just like you described here

       /**
        * Now, we will take the context object that we just created in the `tags` loop
        * and add it to the `file.data` object of the template. We _could_ instead pass
        * the object to the render method, but passing it on `file.data` ensures that it
        * will be used as context on this template only.
        */

Let's not focus on the building of the tags object since I want to make the plugin more versatile but moreso focus on the process.

Within the through2 flush function

// get template
var tmpl = assemble.views.indices[opts.template];
// does this make a copy of the indices template? or is this why the previous `.indices(glob)` call?
tmpl.data = {items: ..., index: ...};
// now you said not to use `.render()`
this.push(tmpl); ? 

One more thing: Shouldn't we also pull in session.get('src-opts') in the options variable?

Looking forward to your comment. BTW, If this skeleton is 'set in stone' once, I think afterwards is going to be relatively easy.

[jonschlinkert]

Because files is ours {} and not the stream. Also,

oops, I forgot to re-add this.push(file) below files.push(file). sorry about that. still reading the rest

[vwochnik]

Further, I would 'forget' about relative links since the template (index template) gets access to the data with the file destination and there the user can either make a relative link or an absolute /link/to/post.html.

Another thing: I saw that isPartial: true is default for whatever reason. So in the collection we create, state isPartial: false?

Also, what do you have with your tags? :D

Index for me

• page 1
  • item 1 ... 10
• page 2
  • item 11 ... 20
    ...

Do you want:

• page 1
  • first 10 items grouped by tag 'xyz'
• page 2
  • five items grouped by 'xyz'
  • five items grouped by 'abc'

? I think that this is unimportant now but can be accomplished by a grouping function somewhere. Once all data objects are collected, they can be arranged and grouped how ever desired.

[vwochnik]

A little more detail:

Plugin does this:

  // introduce renderable `index` template collection
  assemble.create('index', 'indices', {isRenderable: true});

User does this:

assemble.indices('templates/indices/*.hbs');

User does this:

assemble.task('posts', function() {
  assemble.src('templates/posts/*.hbs') // 'layout' can be specified in src.options
    .pipe(index('posts', {limit: 10})) /// but 'layout' can be overridden here in options as well
    .pipe(assemble.dest('dist/'));
});

Plugin collects all file.data objects in first function.

In second function, plugin does this:
options = {} < defaults < assemble.defaults < session.src-opts < plugin-opts
data is list of data (can organize whatever unimportant now)

This is the first parameter. It's not a glob since we're using only one source template but rather a previously loaded template
var tpl = assemble.views.indices[template];

Plugin attaches data
tpl.data = ...

But should plugin clone view first?

How to render afterwards? .render() not necessary?

[jonschlinkert]

But I've specified in the user docs that the user has to do it himself just like the layouts:

We just need to make sure that .indices() or .index() actually exist when the user define them. Using it in the plugin ensures that the templates will be loaded, but... the user can still do what you defined in the docs as long as it's defined inside that task's function (closure).

cb(null,file);s straight from the docs!

It is? where? ah wait I just looked, it's here https://github.com/rvagg/through2#options. But, note that the vast majority of gulp plugins I've seen use this.push(file). we try to follow gulp convention.

You also say {layout: 'page'} in the options.

layout is a different concept. Layouts are applied at a different point in the build and this option allows the user to define a default layout to use for all templates.

is it possible if the user put this .pipe() before .assemble.dest that the templates passed through are rendered

yes, you can do it that way. which reminds me why I didn't push the files through in the main function, since after the dest we only want the index page to render. I just wanted to isolate the functionality to what we're doing.

Let's not focus on the building of the tags object since I want to make the plugin more versatile

These are the guidelines we follow for plugins. My recommendation is to just choose a specific thing for the plugin to do, obviously it's not tags, that was just a convenient example that I think makes sense for this. This way you could easily abstract out the different features into plugins, and if you wanted to you could generalize the logic into utils that can be shared across those plugins. Since the transformations are done in-stream, any impact on build-speed from using separate plugins is minimal. the model works pretty well.

Shouldn't we also pull in session.get('src-opts') in the options variable?

This is already taken care of in stack.js.

and there the user can either make a relative link or an absolute

yes, true. this is what I'm doing in the example. I like that approach better too.

I saw that isPartial: true is default for whatever reason

I can't think of a reason not to have it true by default, since it allows any template to be included injected into other templates. There are lots of use cases for adding relative links/lists to other pages, rather than generating an index page.

I think afterwards is going to be relatively easy.

👍 yeah, I think this is why I like focusing on one specific thing first, then I create a couple/few projects based on the same pattern it becomes more clear where logic should be generalized.

let me know if I missed anything!

[doowb]

A couple of comments:

But should plugin clone view first?

If you're going to do the pagination and push each page into the stream, then you need to clone the view and update the cloned data with information you want for each one. Otherwise, the updates will change the data on the original view.

This is already taken care of in stack.js.

This is only going to happen if you actually use the plugin before assemble.dest and push the new templates onto the stream. If you decide to use .render after assemble.dest and write out the files in the plugin, then you should merge in the src-opts before rendering.

I would 'forget' about relative links since the template (index template) gets access to the data with the file destination and there the user can either make a relative link or an absolute

This is true when getting relative links in the index template, but if you want to build up a data collection containing links to the index pages so other pages can use them (like showing a list of tags in the right column), then you'll need to queue up or buffer the files (like doing files.push(file)) and push those back into the stream in the flush function after you can guarantee all the data has been collected.

I think that's all for now.

[jonschlinkert]

@doowb do you want to do a new example repo that covers the things you mentioned, like pagination, since mine doesn't cover any of that?

[vwochnik]

Okay, thank you. Just a couple more:

1. how do I clone a view that I got via this?
   js var tpl = assemble.views.indices['whatever'];
2. How do I push these onto the stack in such a way that they can be rendered to a file or be used in different plugins?
3. Btw, @doowb if you have any code concerning pagination (yet) I'd appreciate it if you post a repo or gist.

[doowb]

I forked @jonschlinkert's example repo and added pagination to it before realizing that @jonschlinkert is just showing how to list a collection of items with associated pages. I viewed index pages as a way to show paginated items with their list of pages and also showing paginated pages per item. I can see how they can go together but I'm having a hard time completely decoupling the ideas (when talking about index pages).

The changes I made are here: https://github.com/doowb/assemble-tags-collection-example
but it's still limited in how the destination paths are calculated and used as relative paths.

In the example I didn't have to clone the view because I realized that the other way to do it is to split the data out and pass it in as locals to assemble.render. This allows rendering the same template with different data. Each page is given a unique file path and pushed back into the stream so it's written out normally.

As @jonschlinkert mentioned, it's best practise to keep plugins simple so a lot of those functions should be split out into modules that can be used in other places and more easily maintained.

[jonschlinkert]

I didn't have to clone the view because I realized that the other way to do it is to split the data out and pass it in as locals to assemble.render

did you test that? is that advantageous over putting in on the file.data object of the file that is created in the flush function?

I viewed index pages as a way to show paginated items with their list of pages and also showing paginated pages per item. I can see how they can go together but I'm having a hard time completely decoupling the ideas

Imagine 5 plugins in a row only build up collections, the last one generates the actual pages and pagination.

[doowb]

is that advantageous over putting in on the file.data object

That file object is being created after the content has been rendered using the data.

Imagine 5 plugins in a row only build up collections, the last one generates the actual pages and pagination.

I was just thinking that when thinking about how things could be split out. This is where you wouldn't want to render the files in the collection plugin and just add them to the views as a cloned object:

var clone = require('clone-deep');
var tmpl = assemble.views.indices[template];
var file = clone(tmpl);
file.data.tags = buildLinks(tags, files);
stream.push(file);

This is assuming that you're reusing the index template. I think @jonschlinkert was intending that the index template would only be used once for the collection you're building (e.g. tags). If I'm wrong, please correct me @jonschlinkert.

[vwochnik]

When I do this.push(tmpl) no matter whether with the original or cloned template, I'm getting an exception.

/Users/Vincent/Workbench/assemble-v060-test/node_modules/assemble/node_modules/a
ssemble-utils/node_modules/session-cache/node_modules/continuation-local-storage
/context.js:78
      throw exception;
            ^
TypeError: Arguments to path.join must be strings
    at path.js:360:15
    at Array.filter (native)
    at Object.exports.join (path.js:358:36)
    at DestroyableTransform._transform (/Users/Vincent/Workbench/assemble-v060-test/node_modules/assemble/lib/plugins/paths.js:30:29)
    at DestroyableTransform.Transform._read (/Users/Vincent/Workbench/assemble-v060-test/node_modules/assemble/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:184:10)

That's probably because assemble.view.*.* objects are not a file is it? When I push a new File(), it is working properly but I wanted to try this without the .render() method.

I've replaced the .render() block with this:

      var file = clone(tmpl);
      _.extend(file.data, locals);
      this.push(file);

[doowb]

I forgot we haven't made some of the updates in assemble that we made in verb yet. Currently, the objects on assemble.views.*.* are not vinyl files.

I read through some of the previous comments and I think we got off track from the original idea.

The main idea is to generate multiple pages based on a list of data, right? The list could be pages, posts, or something else (tags).

Should the index/indices views just be the source of the template to use for all generated pages or
should the assemble.views.indices contain the template objects that are the generated pages?

If the indices are just the source templates, where should we add the generated pages?
If the indices are the generated pages, where should the source template come from?

@jonschlinkert I think these are the types of things template-utils methods are good for.

I think I got confused somewhere in these comments and I'm just trying to understand what we're solving here (collections or paginated list pages).

[vwochnik]

I've probably already asked too many questions, but please spare your time once more and look at this code:

https://github.com/vwochnik/assemble-plugin-index/blob/master/lib/index.js

I know the pagination handling and stuff isn't great, but please look at:

• Use of template-utils
• Error handling (so far)
• Rendering within the recursive worker function

I said I want to create a plugin and i will! I'm looking forward to the v0.6 release!

[jonschlinkert]

Great! I'll take a look when I get online

Sent from my iPhone

On Apr 1, 2015, at 11:23 AM, vwochnik [email protected] wrote:

I've probably already asked too many questions, but please spare your time once more and look at this code:

https://github.com/vwochnik/assemble-plugin-index/blob/master/lib/index.js

I know the pagination handling and stuff isn't great, but please look at:

Use of template-utils
Error handling (so far)
Rendering within the recursive worker function
I said I want to create a plugin and i will! I'm looking forward to the v0.6 release!

—
Reply to this email directly or view it on GitHub.

[jonschlinkert]

wow, that looks great! sorry been super busy.

I'll pull the code down and run through it as soon as I have a chance, then I might have more feedback. At first glance though, the only thing that might be good to change is the name of the .page() method on Paginator. might be better to use something like .item(), since pagination isn't limited to pages (and page and pages are already assemble methods

Honestly for being new to these libs and assemble v0.6.0, I'm impressed at how many of the nuances you picked up on!

[vwochnik]

I've seen other people moan about classical v4.2 collections. Would you consider it a good idea to split asemble-plugin-index into a plugin to collect and put data into assemble and the index plugin to take the collected data? Instead of .pipe(index()) you'd have .pipe(data()).pipe(index()) ?

Btw, with assemble.data() I can specify files to contain data but how do I actually access the data during the pipeline? Say I want to insert data and later extract it by another plugin how is that done?