protobuf operation

quite impressed by protobuf performance, that the built in implementation is even better than streaming earlier:

models.toBuilder().clear().mergeFrom(
                Files.list(..).filter(Files::isRegularFile)
                        .map(Path::toFile)
                        .filter(file -> ...)
                        .collect(Collectors.toList())
                        .parallelStream()
                        .map(file -> ..)
//read into protobuf
                        .reduce((m1, m2) -> m1.toBuilder().mergeFrom(m2).build())
                        .orElseGet(()-> Stress.StressModels.getDefaultInstance())
            ).build();

models.getModelsList().parallelStream().forEach(...)

is even better performing than

models = Files.list(Paths.get(..)).filter(Files::isRegularFile)
                    .map(Path::toFile)
                    .filter(file -> ..)
                    .collect(Collectors.toList())
                    .parallelStream()
                    .map(file -> ..)
                    .flatMap(m -> m.getModelsList().stream());

models.parallel().forEach(...)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s