Facebook tries a new way to release open-source projects

Last week, Facebook launched Create React App, a new project that helps React developers get started with their new projects. Turns out, that was only part of the story. Create React App was also the first project to enter the Facebook Incubator on GitHub.

The Facebook Incubator is the company’s new process for releasing open-source projects and ensuring that they do well in the long run. The best way to think of it is as a beta stage or proving ground for new open source projects from Facebook.

As Facebook’s head of open source James Pearce told me, the idea here is to better manage the life cycle of these projects. He notes that Facebook has now open-sourced almost 400 projects and has hundreds of thousands of followers on GitHub. “We want to make sure we are managing this program at scale in the most effective way we can,” he said. To do that, Facebook decided that it would push most new projects through this program first to see how the community reacts to them and what the adoption is like.

Pearce stressed that all of the projects in the Incubator — just like in Facebook’s top-level repository — are projects the company also uses internally and that have teams actively working on them. You shouldn’t think of projects in the Incubator as a repository for weaker projects, he noted.

To graduate from the Incubator, projects will of course have to demonstrate traction in the community, but Pearce told me that the company will also look at other surrounding aspects. Is the project being used by others? Does it have good documentation? How hard is it to integrate the project with other tools? How engaged can Facebook be with the community?

“If we see there is resonance in the industry, it’s a good sign that it’ll graduate,” he said.

Pearce did stress documentation is an important factor at various times during our conversation, and that’s definitely an aspect of open source that is often neglected. He told me that Facebook has a dedicated team of tech writers who work on this for its projects (with engineers helping out as well) and that the company is also looking at the new Stack Overflow Documentation service for potentially hosting some of its documentation projects, as well.

While the Incubator is clearly meant to help get projects started on the right foot, Pearce argued that it’s not just about optimizing for the launch and growth phases but also about managing the life cycle of a project in the long run.

Not every project turns out to be a success, after all, and occasionally Facebook ends up sunsetting some of the tools it open-sourced. That will still happen now that the Incubator system is in place, but the team obviously hopes it will be able to correct some of the issues with a project before it moves to the main repository.

Pearce told me that Create React App is a good example for a project in the Incubator because Facebook wasn’t sure what the community would think about it, but he also noted that there will still be some projects that will skip the Incubator project.

“Had we launched React Native now, we probably would’ve skipped the Incubator,” he said. The same goes for projects that Facebook is donating to larger organizations like the Open Compute Project.

Pearce tells me that the Incubator isn’t going through its own incubation phase (“that’s too meta for me”), so we can probably expect this new system of releasing open-source software from Facebook to stay in place for the foreseeable future.

At Carnegie Mellon, using tech to make teachers more engaging

Amy Ogan, an educational technologist at Carnegie Mellon University, calls herself a “CMU lifer” and for good reason. She nabbed both her undergraduate degree and Ph.D from the school. For the last couple of years, she has also worked at the university as an assistant professor, where she’s primarily focused on making classrooms, both online and offline, far more engaging.

Earlier this week, we talked with Ogan about how her team of researchers is right now trying to make good old-fashioned university settings more compelling through what they call “sensing.” Our chat has been edited for length.

TC: What inspired you to examine real-world classroom engagement?

AO: One of the motivations of this work is that with [online courses], we can collected data on what students are doing all the time, but there’s a lot that you can’t see, and a lot of those things demonstrate learning and give us better feedback. So we’re taking the idea back to the physical classroom first.

TC: You’re collecting talk data to start. What kinds of patterns are you looking for?

AO: The first thing is to just look at who is talking in the classroom, which is a major indicator of how students are doing — and how the teacher is doing, too. In many university classrooms, it turns out that not a lot of active learning is happening.

TC: What tech are you using to collect this data?

AO: We’re using sensors that aren’t expensive and can be set up in a variety of classrooms. You don’t need to build a million-dollar, state-of-the-art classroom. It’s microphones and cameras.

TC: What in the talk data tells you that students are engaged?

AO: We’re looking at the frequency of questions, but also how often the instructor is pausing. It’s been demonstrated that if you wait just three seconds after asking a question, you get orders of magnitude more participation from students in the class.

In fact, one approach we’re trying is when we detect that the teacher has been talking too long without interruption, we flash a big red screen on his or her laptop that sort of breaks that lecture mode and gets them to stop and solicit student participation.

TC: How do they react?

AO: They’ve been reacting favorably to it. The trick is not to interrupt the teaching in a way they can’t handle.

TC: And what are you doing with this analysis?

AO: We’ve figured out that the most productive approach is to incorporate some of this information into weekly training exercises for teachers and teaching assistants. One training exercise might prompt them to ask open questions that let students explore ideas, versus closed questions that check whether students have read materials. Then we look at week’s end to see whether they asked those open-ended questions.

TC: Who, or what, is poring over all this data?

AO: A research team here is working on it every day. But we’re working to  verify that you can use machine learning to do this in a completely automated way, as if a human was analyzing the data.

TC: This is a research project, but if and when it’s time to commercialize it, what do you think the product might look like?

AO: It would be easy for any teacher to set up; they’d receive feedback in an automated fashion. They’d receive training exercises on a weekly basis that help them gauge how they’re doing. And they’d hopefully improve their own teaching and get better over time.

While we’re focusing on talk first, we’ll study facial reactions, too. A number of studies has shown how posture is related to student learning experiences, as well as how many people have their hands raised.

TC: You’re not alone in trying to measure student engagement. What’s your biggest differentiator?

AO: I’ve seen a number of desktop -based systems that feature teacher dashboards, so teachers can look at the work that students have done across classrooms and determine, say, maybe the most common error that students are making. The idea is a similar one [to ours] in that it allows you to collect data so you can make improvements. But [those existing systems] don’t show data about the teachers to better support learning. That’s really the piece that we’re trying to tackle — changing the way people approach teaching to turn lectures into more active participatory learning environments.

Why open source is growing – and dying – at the same time

To be honest, if the greek king Sisyphus would be a developer writing open source code in 2016, he would feel at home.

Sisyphus’s famous punishment, handed down by the gods, was to be forced to roll an immense boulder up a hill – only to watch it roll back down after reaching the top, repeating this action for eternity. Almost without noticing, the world’s developer community took upon itself a similar punishment over the past few years. But now the boulder keeps getting bigger.

The U.S. Library of Congress holds around 24 million catalogued books. By many measurement, it is the largest pool of written human knowledge ever created by mankind, the history of thousands of years.

In 2009, GitHub was founded. It now holds over 35 million libraries, or repositories, holding dozens of trillions of lines of code. Studies show that this amount is growing at an exponential rate and has doubled in size every 14 months or so. Open source code is without a doubt the cutting edge of today’s programming technology and is one of the greatest, most powerful and most advanced stockpiles of knowledge ever held by man. Shiny new frameworks are industry benchmarks and brilliant open source builders are rockstars.

So, how come 90%-98% of all open source code is thrown away after 12 months?

The code is in the details

Here are some startling numbers: a year after the day they were first written, over 90% of repos will never be touched and never be used again.

They become inactive and obsolete, forgotten in the sands of time. On its 2015 survey, Stack Overflow found that the average developer spends roughly seven hours a week programming outside of work. GitHub reports having over 12 million users working on open source projects. Humanity is throwing away and tossing aside millions of working hours spent by millions of bright people.

The crazy part? No one seems to asking “why?” Why is the vast majority of written open source code buried and forgotten? Why are we writing the same code over and over again every single day, when at the same time this code almost certainly exists somewhere around the open source – just waiting to be used? 

It is happening mainly because people treat repositories exactly like that – as repositories. Everyone knows AngularJS, or JQuery or React, but very few people know more than ten open source packages. And that is the crazy part – because people don’t know of, or don’t use, the entire package, no one uses the code within it. A package written in 2015 might not be useful to someone on the whole, but perhaps it contains just the function needed for something else. The most useful parts are not always the entire packages, but sometimes the code pieces within them.

Let’s say someone is looking for a JavaScript function to shuffle elements inside an array, or a different function to create a random string of characters. Those small code pieces exists hundreds of times throughout the open source. But no one knows they exist, and even if they did, no one knows how to find them. So priceless amounts of valuable knowledge are discarded or forgotten, just because they are not accessible. This is insane, and it’s bad for everyone.

Organize all code and make it easily accessible

So do we solve this mess? Easy to answer, tough to do – you need to do three things:

1. Organize all open source code by functional pieces: Functions, Libraries, etc.
2. Build a model to represent what each and every one of those different pieces actually do, as in “what is their functionality?”
3. Create an easy and simple way to search and find those pieces of code.

This is why we built Cocycles. It is all of the above, but is also a work in progress. Its algorithms process a huge amount of open source code, reading through it and understanding the functionality of every different line, function or other functional unit. It then allows people to search for that code using plain English.

In the example mentioned above, a user will only need to type “shuffle array” or “create random string” and then they’ll be presented with a variety of open source code implementations, documentations, usage examples and more. It will even offer to generate a useful snippet already containing all dependencies and sub-functions.

In the future, years from now, AI software might be able to use this to find and learn new code by themselves, allowing them to evolve, change and grow on their own. But right now, it currently only supports Javascript, and it is a free and open technology built to make sure the exponential growth in open source code is met by a growing ability to share and use that code.

The author is head of growth at Cocycles.