ng test: Collect total code coverage in multi project repository #11268

Activity

1. clydin

   added

   featureLabel used to distinguish feature request from other issuesLabel used to distinguish feature request from other issues

   on Jun 18, 2018
2. dtychshenko commented on Sep 6, 2018

   

   dtychshenko

   on Sep 6, 2018

   More actions

   I've ran into a similar issue today. Our Angular workspace has one app and four libraries, each producing their own coverage results. I wanted a combined HTML report after all tests for all projects have run. I'm not sure if there's a better way, but so far I ended up with the following setup that seems to be working so far:

   1. karma.conf.js file in each project is configured to report to its own project folder and produce a json report:

       coverageIstanbulReporter: {
         dir: require('path').join(__dirname, '../../coverage/libraries/<project>'),
         reports: ['json'],
         fixWebpackSourcePaths: true
       },

   2. Each project has an NPM script configured to run tests with code coverage:

       "cover:library:<project>": "ng test <project> --code-coverage --watch=false",

   3. There is a high level NPM script to run all tests for all projects in parallel using npm-run-all library (we're on Windows here):

       "cover": "run-p cover:**",

   4. There is a post NPM script to combine the results of all test runs into a single JSON report and a small script to produce the actual reports that I want from the combined JSON:

       "postcover": "istanbul report json && node combine-coverage.js",

   5. The combine-coverage.js file looks like this:

   const createReporter = require('istanbul-api').createReporter;
   const istanbulCoverage = require('istanbul-lib-coverage');
   const coverage = require('./coverage/coverage-final.json');
   
   const map = istanbulCoverage.createCoverageMap();
   Object.keys(coverage).forEach(filename => map.addFileCoverage(coverage[filename]));
   
   const reporter = createReporter();
   reporter.addAll(['html', 'lcovonly', 'text-summary']);
   reporter.write(map);

   This way, when I execute npm run cover, I have tests running for all projects in parallel and get a unified coverage report.

   The last step was necessary in order to produce a newer-style report with correct source paths and styles. At first, I was just running istanbul report html as a post command, but that is using an older Istanbul CLI and uses an old built-in HTML reporter which was producing a report with absolute source paths and broken styles. Going through the istanbul-api in the script makes use of the latest istanbul-reports that produce the correct HTML output.

   Hope this helps

   👍React with 👍14Reacted by FabianaFaria-awin, Remi Vledder, Alexey Kostevich, Dmytro Mostovyi, mihalcan, Ben Sorohan, Felipe Jaramillo , Domas Trijonis, iAnt, Charlie Martínez and 4 more

   ❤️React with ❤️2Reacted by Sergiu Bologa and Miquel Ferrer Llompart
3. Annie-Huang commented on Feb 28, 2019

   

   Annie-Huang

   on Feb 28, 2019

   More actions

   What is content in the coverage-final.json file? @dtychshenko
4. dtychshenko commented on Mar 1, 2019

   

   dtychshenko

   on Mar 1, 2019

   Last edited by dtychshenko

   More actions

   What is content in the coverage-final.json file? @dtychshenko

   There are multiple coverage-final.json files. Here's how it works.

   coverage-final.json file is produced by Istanbul and simply contains code coverage details in JSON format. In the above configuration, a coverage-final.json file will be generated for each individual project because we've requested reports in JSON format (reports: ['json']) in the Karma config of step 1.

   In step 4, the first command istanbul report json will read each of the individual coverage-final.json files for each of the projects and will produce a single combined coverage-final.json report file.

   The script in step 5 reads the last combined JSON report to produce 'html', 'lcovonly', 'text-summary' reports out of it.

   Note that your mileage may vary with different library versions, but this setup currently works for us with the following:

   +-- @angular-devkit/build-angular@0.11.0
   | `-- istanbul@0.4.5
   +-- @angular/cli@7.1.4
   +-- istanbul-api@1.3.1
   | +-- istanbul-lib-coverage@1.2.1
   | +-- istanbul-lib-instrument@1.10.2
   | | `-- istanbul-lib-coverage@1.2.1  deduped
   | +-- istanbul-lib-report@1.1.5
   | | `-- istanbul-lib-coverage@1.2.1  deduped
   | `-- istanbul-lib-source-maps@1.2.6
   |   `-- istanbul-lib-coverage@1.2.1  deduped
   +-- karma@2.0.5
   `-- karma-coverage-istanbul-reporter@1.4.3
     `-- istanbul-api@1.3.1  deduped
   

   I see that @angular-devkit/build-angular in its latest versions has removed the deprecated istanbul@0.4.5 dependency, so the istanbul report json command might no longer work for you, but it's not a big problem.

   All you have to do in your combine-coverage.js script is instead of requiring the single combined JSON report file ( const coverage = require('./coverage/coverage-final.json');), read each of the individual coverage-final.json files in each of your packages.

   👍React with 👍3Reacted by Wandrille, Owen Buckley and Dirk Thierbach
5. charliemc commented on Sep 25, 2019

   

   charliemc

   on Sep 25, 2019

   More actions

   Thank you @dtychshenko for sharing your approach, it really helped here!

   Regarding the removal of the istanbul@0.4.5 dependency in @angular-devkit/build-angular I wanted to share in case somebody else finds it useful that I was able to generate a full report straightaway using istanbul-combine. All single reports generated previously will be combined and merged by istanbul-combine into a single one.

   So, in this case, steps 4 and 5 could be replaced by a script of this kind:

   "postcover": "istanbul-combine -d coverage/reports -r lcov -r html /coverage/libraries/*.json"
   

   👍React with 👍4Reacted by Dmitriy Tychshenko, Dmytro Mostovyi, Owen Buckley and Lukas Ettenauer
6. matses commented on Jan 28, 2020

   

   matses

   on Jan 28, 2020

   Last edited by matses

   More actions

   I was able to generate a full report straightaway using istanbul-combine. All single reports generated previously will be combined and merged by istanbul-combine into a single one.

   @charliemc Could you post your package.json & karma.conf.js ?

   i get some known issue with istanbul-combine :

   gotwarlost/istanbul#817

   Update : fixed with --coverage --source-map=true
7. alan-agius4

   added

   devkit/build-angular:karma

   area: @angular-devkit/build-angular

   on Jan 11, 2021
8. ngbot

   added this to the Backlog milestone on Jan 11, 2021
9. wkit23 commented on May 31, 2021

   

   wkit23

   on May 31, 2021

   More actions

   For those who still needs this, I manage to get the report to work with the following script. I used karma-coverage to return json format which then the script will merge and combine into other reporting format.

   Would be great if we can do this directly from the cli.

   var glob = require('glob');
   var libCoverage = require('istanbul-lib-coverage');
   var libReport = require('istanbul-lib-report');
   var reports = require('istanbul-reports');
   
   var map = libCoverage.createCoverageMap();
   
   // Gets all the coverage files and merge them
   var files = glob.sync('./tests/coverage/**/coverage-final.json');
   files.forEach(f => map.merge(require(f)));
   
   // Creates a context for report generation
   var context = libReport.createContext({
       dir: './tests/coverage',
       coverageMap: map
   });
   
   var reportTypes = ['html', 'lcovonly', 'text-summary', 'cobertura'];
   reportTypes.forEach(t => {
       var report = reports.create(t);
       report.execute(context);
   });

   👍React with 👍1Reacted by Rafael Guimaraes Siqueira

   😕React with 😕1Reacted by Leonardo Chaia
10. angular-robot

    added

    feature: under considerationFeature request for which voting has completed and the request is now under considerationFeature request for which voting has completed and the request is now under consideration

    on Feb 1, 2022
11. ngbot

    modified the milestones: Backlog, needsTriage on Feb 1, 2022