During our last Bay Area Gradle Users Meetup on May 26th, 2016 @ App Dynamics, Hans Dockter gave a demo of all the new features you will find in Gradle 3.0:
– Daemon enabled by default
– Composite Builds
– Performance Improvements
Watch the full video here.
Video Transcript
Yeah, I want to talk about Gradle 3.0 and that is a big topic because a lot of fundamental stuff is happening. We’re super excited about what is coming and let’s start with the Gradle Daemon. Who of you is using Gradle with the Daemon. Great, and who of you had ever to kill the Daemon. One thing we will do in Gradle 3.0 is it will be enabled by default. Right, and that puts some pressure on the Daemon to capture some cases where it might be causing trouble and confusion, right, to our users. At the same time, even though it’s very easy to enable the Daemon, especially with the non power Gradle users most do not enable Daemon. That is a big performance penalty as you all know, so in a way it’s for a big part of the community.
It’s the biggest performance improvement we can do, is enable the Daemon by default so they don’t have not to do it. One part is, what we want to do better, is tell you more about what is happening with the Daemon. Let’s … I’ll make the fonts bigger. There’s some five hundred sub-module build and let’s say I’m gonna start that build and first the messaging is getting better. The Daemon has started and let’s start the Daemon again. Sorry, let’s build a project. The same project in a different tab and what you see now here, another Gradle Daemon is busy, so that’s why we’re starting a new Gradle Daemon. That has happened before, but you had no idea what was going on right? Another thing that you will see is, let’s do something like Gradle classes dash D org, Gradle, JVM aux, [inaudible 00:02:42] oops.
What I wanted to see is … Let me quickly check something, that’s correct what did I do wrong? What I wanted to show you is a different message. Why is this not happening? I wanted to see the message that this is incompatible with the running Daemon and that’s why we are starting a new Daemon. Not sure why this is not happening. I think the arguments are correct. Okay, an idle Gradle Daemon with different restraints could not be used because different memory allocation for the Daemon, right? Even if it’s the same, we theoretically could use the same Daemon, but you want different memories so we tell you,”Hey there’s another Daemon running, we can’t use it because different memory constraints.” Another thing that is interesting is … Let’s run this Daemon with a hundred twenty-eight memory and you should, something should happen.
It’s already slow because of the memory situation. Daemon is stopping because the JVM tenured space was exhausted. Right? This is brand new, this is master, but it’s not stopping that’s interesting. Maybe because it’s already happening at configuration time. Okay, anyhow this is a non release version so just that you get an idea of what is happening. Here we go. Now we get an out of memory is happening, exactly what we wanted to prevent. Anyhow, it will work once M1 is out. The key problems we see with the Daemon is that people don’t have a good idea about what is going on, especially developers that are not the build masters.
We want to tell them what is happening. At the same time we want to also basically proactively react. When things are getting close to out of memory, instead that people wondering what is going on, why is this compile now taking, I don’t know one hour we just say, “Hey,” we stop the Daemon and tell you, “You have a memory issue.” Right? That is much better than, “What is going on, why is this built so extremely slow?” It’s a lot of good stuff happening to basically to provide a good user experience when the Daemon is enabled by default. The other thing I’m very excited about is a feature called composite builds. I’ll go immediately back to demo mode. What we have here, is if you look at projects, you see A, B, and C. Those projects are completely independent. This is not a multi-module build. Project A, project B, and project C are different builds right? If you look for example into project B, right, you see project B is actually a multi-module build. With two sub-projects B1, and B2. What if you want to build them together? What if you need to work on project A, and on project B at the same time, and project A depends on project B right? You do some changes in B and you want that A is picking them up. What would you do currently? [inaudible 00:06:35] yes, or do something like run project B, publish it to a repository, and then project A is picking up the changes. It’s very, it’s not convenient right? But we now have composite builds. It’s still work in progress but it’s already very excited what I can show you. We have now … The root pro, if you like the root of this, it’s just … This erratic construction is just for convenience purposes.
Right, they don’t have to be in one directory structure. This is now a composite project where you can define different Gradle builds to be built as a composite. When I now say, “Gradle show me the dependencies of that composite.” Something interesting is happening. You see project A has a dependency, right on B1, and B2. This is automatically replaced with the project dependency of project B, B1. Right, so Gradle understands automatically, “Okay if that is the composite, I scan all those different builds whether they have an external dependency on one of the other builds. If they have, I change the dependency set up. Instead of retrieving the external dependency from a repository, the other project I depend on will be built and the output would be injected into that project.” The way that will be working eventually is you have a warning here that, “All participants will be executed in a single Daemon process.” Right? Eventually those builds can have even different Gradle versions, and it will work right. Different Daemon settings, different whatever. The ID coupled, but still the output and the input relations are honored and you can work with them as if they were one build. This works already to set it up on idea. Come on, good file. Could not execute Gradle installation. [inaudible 00:09:24] let me check something.
No idea what’s going on here, another brand new feature. I usually switch to pre-recording my demos, now i know why. You have the project here, right, in idea. I did it before, and what you can see here is that idea will automatically … Creator will automatically create a metadata for idea that project A is not depending on external dependencies, but that they are translated to module dependencies in idea. You can work on that as if it were one code phase, re-factor and rework, etc. As soon as you exercise the build outside of the composite, it will be completely able to run, stand alone, and retrieve external dependencies. This is one important piece to make multi-project scenarios much more convenient for developers. You have basically a fluency to decide when you have multiple repositories, “I want to work on multiple repositories at the same time,” or “I want to work on them in isolation.” The road map also … One important step for the road map is you can even execute the composite on different machines. It would be our entry into distributed builds and distributed executional builds. I think it’s a nice granularity.
It scans all those builds right? There’s a pre-processing phase where we scan those independent builds and say,” Hey what are your dependencies?” Then we see, “Oh just look at project A at the build file.” One second. Let’s go to project A, and you see this is just an external dependency on aux sample B1, one not zero. The B project has a group ID, it has this group ID, so we know … And the artifact that is, so we know this project corresponds to that external dependency and then we have mechanisms in Gradle to replace that. To ignore basically that this is external dependency and substitute it with project dependency that is built locally. Another area where we have made massive improvements over the last three months, four months, is performance. Who of you is already using Gradle 2.14? Well, the release [inaudible 00:12:18] Who is using Gradle 2.13? With 2.13, you saw already some nice performance improvements, but they will be even more significant with 2.14. For many builds, we kept the configuration time at least by a factor of two. For a few builds by a factor of three and more. That’s pretty spectacular right?
The configuration time improvements, we are not at the end of that effort, but it’s absolutely substantial. If you have a chance Gradle 2.14 release candidate one is out. I think tomorrow we’re gonna release candidate two, so 2.14 is definitely a release you should pay a lot of attention to because things will be much faster. Another thing we have improved in 2.13, I think, was that script compilation time. Before the scripts are cashed, when you check out a fresh project, compilation is much faster after build scripts. Another thing we introduced in 2.13 is that test execution is much faster. When Gradle runs tests there is a fourth JVM that is running the test, and a JVM that is running Gradle, and there is inter-process communication happening. We sped that up, and we sped up the time to fire up JVM that runs the tests, so that is pretty significant. Especially if you have many modules in your build. It’s easily multiple hundred of processors, so that is in 2.13. What we started to do in 30 is we tapped into dependency resolution time.
When you check out Gradle master, what we did there as a first step in our local cash, we used IV and Pom XML to store the metadata. We have changed it to a binary representation form, which is significantly faster. We saw some significant speed ups in dependency resolution time, and there’s much we’re gonna do in the area of dependency resolution. Cashing across builds, so I’m very excited about that. Especially for android, that is super important. There’s more to come, parallel resolution, making dependency resolution faster when you do parallel builds, so that is an area we’re investing a lot of time, and there’s so much more that we’re gonna do. Performance is top priority, and you can expect, I would say, with every minor release for the next six to twelve months significant improvements. We’re pretty excited about all the stuff we’re going to do to make Gradle very very fast build system. Who of you is building Android applications with Gradle? For Android dependency resolution is even more important because they have all these variants. The number of configurations they need to resolve is much bigger than for typical Java builds. For them, they will benefit most of the improvements here.
The other thing that in our very focused effort to improve the performance is what we have realized that dynamically typed Groovy code, so if you write your plug ins in Groovy, or you have script plug ins that are pretty complex, you pay a significant performance price for that. Depending on the plug in, there is no fixed price I can tell you, but we had a couple of plug ins we are re-factoring dynamic Groovy code to Java made applying the plug in three times faster. There are various things you can do, right? One is you rewrite the plug in, you convert the plug in to Java. That is what we are doing in the Gradle core, but it’s not the most convenient experience. It’s a different APR, it’s you moving from the DSL to the API, right. There’s some un-intuitive stuff you need to do. It’s possible, look at the Gradle code base see how we are doing things, that’s a good example. That is definitely one way to get a much better performance. At the same time, it can be quite involved if you have big scripting plugins, right, you need to now turn them into a plugin project, where they get compiled and all that.
It’s one way to do it, it’s not the most convenient way to do it. The other thing you can do is, Groovy has a not well known feature that is called compiled static. You basically can create Groovy code that is not dynamic. That might, just Google for compile static and you will find that. There is another way to do things, but at the same time it’s not the experience we want to provide to our users eventually. That leads me to the next topic, and a big part of the Gradle 30 story, Gradle goes Kotlin. We are embracing Kotlin as another language to write Gradle builds. I wanted to talk about the motivation for that. First of all, how did that start? It’s kind of an interesting story. I mean, initially I was just impressed. We were impressed with the Kotlin language and with the uptake, especially in the Android community. Who of you has ever used Kotlin? Who of you has ever heard about Kotlin? Okay, so I should then something about Kotlin. Kotlin is a language founded by Andrey Breslov, who is working for JetBrains. JetBrains is the company behind the language, they have twenty engineers working on the language.
It’s aesthetically type language with all the things you expect from a modern language like all the functions, extension methods, and a lot of smart features. At the same time, it is much easier to learn than let’s say Skala. You can learn Kotlin in three days and you will be more productive than you are with Java. It has packed what’s compatible down to Java six, and it has a lot of uptake. I know Netflix is now hosting the Kotlin meetup, right? Many people are interested in that language, and Android has already a strong following. For us, it presents a wonderful opportunity. What is the pain that we’re trying to address with Kotlin writing? First of all, who thinks the Gradle IDE integration is completely perfect and couldn’t be better? Yeah, so no one. There is reasons for that. One key reason is that dynamic Groovy code is very hard to support smartly in IDE. The other reason is that Groovy support for Eclipse is no longer working, there are no resources behind that, so and it’s a real problem. We want to provide amazing IDE support for our users, but with Groovy that would be a major mission, for let’s say our company, to do that or for anyone else.
Still constraint for some of the qualities of the language. The other thing is scaling build launching. I talked already about performance, what you get from aesthetically type languages, and IDE support is also I big part of scaling build launching. You want to have the same code navigation, move from your build script to your custom plugin that you’ve written to the Gradle core. You want to have that navigation capabilities that you’re used to from writing production code. To be able to deal with complex build launching. Stefan talked about that, how many Gradle builds you have. This is a 280 engineer organization and it’s a complex piece of your engineering. Often the domain of building a delivering software is as complex as the business to make. Builds have changed over the last years and we need to address that. A good thing Kotlin, there is just no down side. You can create wonderful DSLs, something you could not do with Java. It’s very scripting friendly. The reason why we picked Groovy over Java years ago as the build language is that it’s just so much more convenient to use for our users. With Kotlin we have more or less the same convenience, but no downside.
An E support that works aesthetically compile code, so we’re very excited about that. When we came to that point, Gradle and JetBrains, we started to discuss, I expressed interest in Kotlin and JetBrains was very excited about that, so we put our heads together and I’ll show you a demo for Gradle Kotlin support milestone 1. What we have here is a simple Java project. A Java application, it has a [inaudible 00:22:54] class, it has a main method and prints some very sophisticated statement. What you’re seeing here is an almost empty Kotlin Gradle build script. Let’s get started. How do you apply plugin again? Ah, with apply, thanks. Things intelligent. What was the name, what is the name of the plugin to build applications with? Ah, it’s the application plugin. Okay, so let’s use the quick documentation look up to learn about the application plugin that goes into the Gradle Java doc. We see, okay the plugin can be configured wired application plugin convention object. Let’s look for the configure method, and let’s look for the application plugin convention object and let’s configure the thing. Now it’s getting interesting. What are the properties of that convention object? Ah, main class name, I need to specify the name of the main class to fully qualified path, spec of samples [inaudible 00:24:19], let’s do that.
That is the Java application that we want to build, the main class for that. Then there are some other properties, application name : what is the name of the application? It’s Kotlin demo, and not going through the Gradle DSL net reference webpage and what not. It’s available, and what I just did, I just clicked Kumon B to jump into the application plugin convention class. I’m now in the source code, and I can do the same for the application plugin itself. It’s seamless navigation from the Gradle source code to your plugin source code base. Now we need some dependencies, right, to run the tests. Room, repositories, main central, dependencies, now it’s getting interesting we’re still working on making the syn techs nicer. That’s how we do it at the moment, test compile, at least the documentation look up works. That’s a dependency for the given configuration. We add the J unit, we want two dependencies. Now let’s see if we can actually run this build. Maybe it just an IDE fake. What we’re doing on Gradle, run the tests and here we go. Kotlin told Gradle what to do. The tests are run, everything is working. We run the application, right, to see how it worked.
We can change that text, we just go to the production code, Kotlin night that was from last week, and we see hello Kotlin night. Let’s do one more thing, we can install this application right and we can now look at … This is the application name, this is the Java for the application, those are the stocks, the stock scripts. Last thing just to show that everything is working, we execute the start script of this application. Here we go. We cooperated with that, with the JetBrains folks, so those four engineers created that. It’s amazing stuff and we’re making progress very fast. If you want to try it out, there is the Gradle script Kotlin repository where you can find the sample I’ve shown to you and you can fully reproduce that. That is available, you can play with it. There’s some inconveniences in there still, but this will all get polished. We speak more about that on our blog, Kotlin-meets-Gradle if you want to get more background. Common questions are, “Will existing plugins with Kotlin Gradle builds?” Yes, whatever language you use to write your plugins, Java, Groovy, Skala, Kotlin it will work. “What about Eclipse?” That’s the the exciting thing, the JetBrains guys also provided Kotlin plugin for Eclipse.
You will get much better Kotlin support on Eclipse than what you have now with Groovy. “Will the usage of Groovy become deprecated?” No, we expect there will be Groovy build scripts for the next five years, so it will not be deprecated and will continue to be supported. Our road map is … This is the beginning, so especially in regard to the DSL, there’s a lot of smart stuff we want to do. For example that you have also content assist for adding dependencies to compile, test compile, configurations and things like that. We are moving very fast, so this is something we want to have production ready this year. Usable, earlier than that. Gradle 30 milestone 1 will have experimental Kotlin support 1.0 milestone 1. What is interesting to understand, the Kotlin support at the moment lives in a different repository so that you can update a newer version of the Kotlin support without being forced to upgrade to a newer version of Gradle. We want to have, for the incubating period, separate life cycles. In 3.0 and 1, it will be embedded, only if you want to update to a newer version of the Kotlin support you need to do something.
Otherwise, Gradle 3.0 and 1 will have Kotlin support out of the box. With production ready later this year, and the detailed road maps will be published over the next weeks. Okay, so super excited about that. IDE support, faster builds, wonderful language that I think has a very bright future. Before I hand over to Stefan, a couple of words to the greater momentum, we’re at 10.4 million dollars in 2015, 1.95 million downloads in April, so last month. 800 plus community plugins growing fast with support for any language and platform, it’s amazing when I look at the plugins so much technologies I have never hear of. They are supported, so we’re growing support for platform Java, Android, CC plus plus, [inaudible 00:30:40], and of course a very good Kotlin support. Generating automation platform to model all aspects of continuance delivery, that is our vision. Okay, thank you and Stefan, looking forward to your talk.
The post Gradle 3.0 Preview by Hans Dockter appeared first on Gradle.