r">

No Description

FFIB: 11e3a9652a first 8 years ago
..
LICENSE 11e3a9652a first 8 years ago
README.md 11e3a9652a first 8 years ago
cache.js 11e3a9652a first 8 years ago
changelog.md 11e3a9652a first 8 years ago
package.json 11e3a9652a first 8 years ago

README.md

file-entry-cache

Super simple cache for file metadata, useful for process that work o a given series of files and that only need to repeat the job on the changed ones since the previous run of the process — Edit

NPM Version Build Status

install

npm i --save file-entry-cache

Usage

// loads the cache, if one does not exists for the given
// Id a new one will be prepared to be created
var fileEntryCache = require('file-entry-cache');

var cache = fileEntryCache.create('testCache');

var files = expand('../fixtures/*.txt');

// the first time this method is called, will return all the files
var oFiles = cache.getUpdatedFiles(files);

// this will persist this to disk checking each file stats and
// updating the meta attributes `size` and `mtime`.
// custom fields could also be added to the meta object and will be persisted
// in order to retrieve them later
cache.reconcile();

// use this if you want the non visited file entries to be kept in the cache
// for more than one execution
//
// cache.reconcile( true /* noPrune */)

// on a second run
var cache2 = fileEntryCache.create('testCache');

// will return now only the files that were modified or none
// if no files were modified previous to the execution of this function
var oFiles = cache.getUpdatedFiles(files);

// if you want to prevent a file from being considered non modified
// something useful if a file failed some sort of validation
// you can then remove the entry from the cache doing
cache.removeEntry('path/to/file'); // path to file should be the same path of the file received on `getUpdatedFiles`
// that will effectively make the file to appear again as modified until the validation is passed. In that
// case you should not remove it from the cache

// if you need all the files, so you can determine what to do with the changed ones
// you can call
var oFiles = cache.normalizeEntries(files);

// oFiles will be an array of objects like the following
entry = {
  key: 'some/name/file', the path to the file
  changed: true, // if the file was changed since previous run
  meta: {
    size: 3242, // the size of the file
    mtime: 231231231, // the modification time of the file
    data: {} // some extra field stored for this file (useful to save the result of a transformation on the file
  }
}

Motivation for this module

I needed a super simple and dumb in-memory cache with optional disk persistence (write-back cache) in order to make a script that will beautify files with esformatter to execute only on the files that were changed since the last run.

In doing so the process of beautifying files was reduced from several seconds to a small fraction of a second.

This module uses flat-cache a super simple key/value cache storage with optional file persistance.

The main idea is to read the files when the task begins, apply the transforms required, and if the process succeed, then store the new state of the files. The next time this module request for getChangedFiles will return only the files that were modified. Making the process to end faster.

This module could also be used by processes that modify the files applying a transform, in that case the result of the transform could be stored in the meta field, of the entries. Anything added to the meta field will be persisted. Those processes won't need to call getChangedFiles they will instead call normalizeEntries that will return the entries with a changed field that can be used to determine if the file was changed or not. If it was not changed the transformed stored data could be used instead of actually applying the transformation, saving time in case of only a few files changed.

In the worst case scenario all the files will be processed. In the best case scenario only a few of them will be processed.

Important notes

  • The values set on the meta attribute of the entries should be stringify-able ones if possible, flat-cache uses circular-json to try to persist circular structures, but this should be considered experimental. The best results are always obtained with non circular values
  • All the changes to the cache state are done to memory first and only persisted after reconcile.
  • By default non visited entries are removed from the cache. This is done to prevent the file from growing too much. If this is not an issue and you prefer to do a manual pruning of the cache files, you can pass true to the reconcile call. Like this:
  cache.reconcile( true /* noPrune */ );

License

MIT

kodo - Gogs: Go Git Service

Nav apraksta

Brightcells: 917f2df489 Add api api/upgrade 8 gadi atpakaļ
..
0001_initial.py df1df69fe3 add api upgrade/splash 11 gadi atpakaļ
0002_auto_20160120_1830.py a121b75ff2 add db_index=True for status field 10 gadi atpakaļ
0003_feedbackinfo.py ebaa78d5ad add api feedback_api 10 gadi atpakaļ
0004_guestentrancecontrolinfo.py 63eaee0951 modify guest_login_api 10 gadi atpakaļ
0005_auto_20160509_1907.py 2f29afecbd modify version in operation 10 gadi atpakaļ
0006_feedbackinfo_src.py a71f3c569b add src for feedback 10 gadi atpakaļ
0007_auto_20160907_1740.py 27d3ec8fb1 add src for LatestAppInfo/SplashInfo 10 gadi atpakaļ
0008_appsettingsinfo.py dd00e3b1a9 add api online_api 10 gadi atpakaļ
0009_auto_20161220_1354.py ead8d009b7 Add src PAIAI_TOURGUIDE 10 gadi atpakaļ
0010_auto_20170308_2242.py 4f87058e1c Add patch_api 9 gadi atpakaļ
0011_auto_20170315_2243.py 2eba3ed7f9 Add guest login for tourguide 9 gadi atpakaļ
0012_boxprogramversioninfo.py afd07e5389 Add api box_program_version_api 9 gadi atpakaļ
0013_auto_20170418_1451.py 24dcbc8f3a Add srv_sha1 & proc_sha1 9 gadi atpakaļ
0014_auto_20170814_2001.py 65db8b1bc1 Add Src PAIAI_LENSMAN2 9 gadi atpakaļ
0015_auto_20170814_2004.py c3970eff70 Add Src PAIAI_LENSMAN2 9 gadi atpakaļ
0016_auto_20180101_2220.py 9bb56c50cc Makemigrations 9 gadi atpakaļ
0017_auto_20180103_0446.py a7cbbf15a7 Update max_length for CharField 9 gadi atpakaļ
0018_auto_20180114_2314.py 917f2df489 Add api api/upgrade 8 gadi atpakaļ
__init__.py df1df69fe3 add api upgrade/splash 11 gadi atpakaļ