Space is a Network.
Constellations is Your Connection.
Transcript:
John Gilroy: Welcome to Constellations the podcast from Kratos. My name is John Gilroy and I’ll be your moderator. Our guest today is Ryan Lewis, Senior Vice President and Lab Director, In-Q-Tel. Today we will discuss how rapid advances in analytics technologies are shaping the broader developments in the commercial space and aerospace markets. New technologies such as artificial intelligence techniques like computer vision, offer new ways to process and analyze the growing volume of remote sensing data from both incumbent technology providers and new venture backed startups.
John Gilroy: In this podcast, find out how the commercial space and aerospace markets have evolved over the last several years and how emerging geospatial analytics applications could change how these products and services are used by public and private sector organizations alike. Our guests today, right in front of me. Ryan Lewis, Senior Vice President and Lab Director at In-Q-Tel.
John Gilroy: He co-founded and leads CosmiQ Works, an applied research lab within IQT, developing machine learning algorithms, open source tools and curated remote sensing datasets for geospatial applications such as foundational mapping. Ryan also co-founded and serves as general manager for SpaceNet LLC, a nonprofit organization dedicated to accelerating open source geospatial analytics, which is run in collaboration with several partners including Maxar, Amazon, Intel, Capella Space, Topcoder and the folks from IEEE GRSS. Well Ryan, it’s very mysterious name here, In-Q-Tel. How’d that come about? What do they do?
Ryan Lewis: Well, first of all, I wanted to say thanks for having me on the show and thanks to Kratos. It’s good to be on the other side of the mic. To start from the top, what is In-Q-Tel and then where does CosmiQ fit into that? In-Q-Tel is a strategic investor for the US national security community. A major milestone for us, was our 20th anniversary this past year in terms of working with both the national security partners as well as venture back startups from across the country.
Ryan Lewis: From the labs perspective, CosmiQ Works is one of In-Q-Tel’s applied research labs that focus on different aspects of artificial intelligence. Specifically, my group focuses on geospatial analytics, as you said in the opening. It’s something that we have been targeting for the last five years and our work has evolved greatly in that time from just preliminary research to essentially building whole open source models that work across a variety of data sets.
John Gilroy: If you’re listening to this and it’s 90 degrees outside and you want to find out more, it’s spelled C-O-S-M-I-Q Works, right.
Ryan Lewis: Correct.
John Gilroy: Good. I know it’s going to happen. People listen to podcast and they’re running or something trying to figure it out.
Ryan Lewis: People should look at the website at least once a week. I highly encourage it. It’s just an opinion.
John Gilroy: A key challenge for the government is getting access to cutting edge technology in the commercial sector in a timely manner so they can apply it. How is your organization helping our government with this problem?
Ryan Lewis: In-Q-Tel was founded with that focus in mind specifically. If you look back into the late 1990s, accessing emerging technologies in Silicon Valley was really challenging particularly if you look at it through the lens of traditional acquisition systems across the government. As a firm, we have really taken our strategical of accelerating access pilots and adoptions to commercial based venture backed startups across the country, across all different types of technology areas. Ranging from software to hardware products.
Ryan Lewis: One of the core ways that we do this is working on an annual basis with our different government partners, venture backed companies and their investors to target what we would call technical work programs. These work programs are specifically designed to help companies that we’ve invested in get over those initial hurdles to accelerate piloting and adoption by our different government partners.
Ryan Lewis: This is a little bit different on the lab side because our work for the most part is operating primarily in the open source, which we’ll talk more about later. But if you think about most of the rapid developments in the AI domain, really in the last five years and one of the benchmarks that we use in the computer vision space is when Google open sourced TensorFlow in November of 15. Ever since then, a lot of the work, at least from an applied perspective that different end users want to get access to for either doing early prototyping or even maybe building out a minimum viable product for testing.
Ryan Lewis: A lot of that starts in the open domain and so whether it’s CosmiQ or some of our other labs, we’ve really tried to focus on building out either open source data sets or tools to help our government partners as well as commercial companies get started with some of these really advanced technologies.
John Gilroy: I think we’re finding that innovation doesn’t reside in the zip code or geographic area. It’s all over the place and I guess it’s a possibility that these new technologies have had some democratizing effect on broader access to public and private organizations. How does that affect your efforts?
Ryan Lewis: It’s arguably one of the most exciting things that I’ve had to experience really in my time, and a key talent and specifically with CosmiQ in the sense that whether you’re talking about a more traditional space systems such as either small satellites or software companies that are focusing exclusively at the applications level. What makes the democratization effect really compelling is that it opens the aperture to either companies or individuals that are contributing either to code or to company ideas.
Ryan Lewis: So, long story short, what that really means is you have a lot more either new starts from companies or interest from investors that simply expands interest in a domain. In this case space that was historically somewhat niche in terms of those providing products and services to it, just given the technical complexity. That’s been really exciting in terms of having new people come in with new ideas and changing in some ways how both the commercial space and now aerospace industry think about delivering products and services particularly at the software level.
John Gilroy: I’m glad you used the word aperture. It’s real popular with this audience.
Ryan Lewis: Yes.
John Gilroy: It should be. Let’s take look at the investment side. The space industry has transformed rapidly with lower cost infrastructure, multi-constellations and more capable satellites. Now, how have these changed your investment strategy in the space/aerospace market?
Ryan Lewis: The story of how we started investing in the commercial aerospace domain is a neat one. It’s one that I wish I could say I could take a majority of the credit for it, but that would be incorrect. Really a lot of the work goes out to some of our partners on the investment side including Tom Gillespie. Essentially what we did about five years ago is we said, all right, we’re seeing changes across the entire market. This is ranging from all the way from access to launch vehicles, all the way to the top at the application space.
Ryan Lewis: We’re having so many new starts as we were just talking about from a variety of different companies. How do we assess this from an investment perspective? In other words, how do we put all the pieces together to build a cogent investment strategy? This is something that you can find on our website under resources. We built what is now called our investment architecture or technical framework. In some ways it’s just a way to arrange different technical offerings and see how to connect together.
Ryan Lewis: But at a strategic level, it allows us to look and say, are we making sufficient amount of investments? Do we have sufficient amount of awareness in different aspects of that stack? Over the years since as companies mature or as we’ve seen in other cases, more money enters into the market for different types of companies.
Ryan Lewis: We’ve modified our strategy over the time. Imagine going back to 2014 a lot of our initial focus was just trying to understand what was happening in the market. Who are the new starts, who are the relevant players, what does this mean from a broader supply chain perspective. Today, that looks very different as companies that have now matured and we’re now more focused on the utility of the products that they’re delivering.
Ryan Lewis: So if you’re a company that’s building remote sensing constellation, it’s much less on let’s look at the satellites or talk about different orbits that you’re thinking about working in and now much more on, show us some of the data, show me some of your early products and tell me where you see some of your product line evolving to in the coming years based on feedback from either early customers or thoughts from your engineering team based on the experiences they had.
John Gilroy: So now the market is probably more mature than in 2014. You cast out the net, you reel in a couple of fish you think are pretty good. Now what? How do you measure them? How do you measure their success or your successful efforts with them?
Ryan Lewis: There’s a couple of different ways to do it and I think at the general level, one of the ways to measure success is just to see how have companies delivered against their strategies. I remember early days. I remember going back and people talking about all the different coverages we were going to have from imagery or other types of remote sensing platforms.
Ryan Lewis: There was arguably some hype around that. What I think has proven very useful for us is now to go back and say, all right, how long did it actually take to deliver some of these assets? Because even though we’ve seen a rapid commoditization in the aerospace domain, it’s still really hard. Elon Musk’s quote is accurate.
Ryan Lewis: As a result that means that some of the timelines may have slipped. That’s not necessarily an indictment on anyone in the company. It’s much more of highlighting how difficult it is to go from an early prototype to a full service solution that companies build and maintain and customers can really leverage.
Ryan Lewis: I think for us, as we look out into the next couple of years, what we’re really watching for now is how does customer demand grow based on the availability of these services.
John Gilroy: Now Ryan, thousands of people from all over the world have listened to this podcast. If you’re listening now, go to Google and type in Constellations podcast. You’ll arrive at our show notes page and it’ll give you free access to transcripts and we can send you email notifications of future shows if you wish. Let’s talk a little bit about open source here.
John Gilroy: When you look at the open source applied research, what motivated you and the team to start CosmiQ and begin open source applied research specific to geospatial analytics?
Ryan Lewis: One of the big motivators for us was actually the foundation of a lot of the early computer vision research in the community. This is not necessarily for geospatial analytics or anything to that effect. If you look at a lot of the early papers that were focused on computer vision, most of them were working on a data set called image net.
Ryan Lewis: This was… It’s been well documented now, but essentially all the images of cats and dogs and things like that and it’s critical. Most of these models started with scene classification. Simple things like that. Just seeing the impact that that dataset and then all the research that came from it had on the broader computer vision research community. We said that is massive, but do we see analogs to that in the geospatial market?
Ryan Lewis: Because in some ways you can certainly transfer a lot of the work that’s been done in this broader computer vision field to different types of remote sensing datasets, no doubt. A lot of companies have done really interesting work, but there are natural limits to it because as you might guess, simply looking at a photo that you’ve taken. For example, taking a photo of us in the studio today, that’s a lot different than saying, “Hey, are there people in this or not?” To then saying, “Find me all the different building footprints at this level of accuracy in a satellite image.”
Ryan Lewis: Those are fundamentally different problems and at some point you need to address those directly, both from an analytic perspective as well as a software perspective. That was really our motivation to say we should be focused on this community to help transition or essentially overcome that gap between what we see in the current research community now and applying it to more geospatially focused problems.
John Gilroy: Now, when you said image net, I wrote an arrow and I said, this is going to lead to Skynet, but it didn’t.
Ryan Lewis: Hopefully not.
John Gilroy: Hopefully not. Talk about open source applied research again. Why do you think it’s so important to the wider aerospace industry?
Ryan Lewis: It’s a great question. It’s one that actually comes up a lot. I think really the simplest way to answer it is that the work that is occurring just more generally across a variety of different AI applications, the level of just product development and acceleration, it’s really unprecedented in the open source domain. This is something that is still hard for me to believe at times.
Ryan Lewis: What I mean by that is it’s a wealth of resources for other companies or governments to get started with their early projects. It is a resource probably going forward for companies to leverage as they’re figuring out how to build their products. The reason to emphasize it is that this is something that is new when you think about how to build out analytic products. Essentially starting with something that’s out in the open and then tailoring it for perhaps a proprietary purpose is somewhat new in this industry. That’s why we really wanted to not just prioritize it but provide almost as a trusted resource for researchers to get started.
Ryan Lewis: What I mean by that specifically is writing rigorous analytic baselines on how we’ve seen certain models perform. Writing detailed analysis on our blog, which you can find through our website. Essentially highlighting what we think is sufficient for certain projects and so on and so forth. We really wanted it to serve as a starting place for companies and researchers when they’re thinking about getting into advanced analytics in the geospatial domain.
John Gilroy: How have you seen the space, aerospace and geospatial analytics sector evolve?
Ryan Lewis: It’s easy to get lost in all the excitement about the sector. One of the best ways sometimes to make sense of all this interest in momentum is to ground yourself in set strategy. One of the things that we’ve done as a team, both on the CosmiQ side as well as on the broader In-Q-Tel side is to set a basic strategy and target our projects towards that.
Ryan Lewis: If you think about some of the early claims in the geospatial domain, we started with a real simple question, which is what effect does, example in this case, what effect does resolution have on model performance? From that you can look at just analyzing inputs from the dataset, resolution, look angle, and then you can look at it from the model perspective. How well do different models perform? Is there a difference?
Ryan Lewis: Essentially, building iteratively across that strategy has been critical for us and so it may not always be the most thrilling to say you’re going to test something for the sixth or seventh time. I know I’ve had more than my share of headaches sometimes retesting, but I can tell you for us it has been, I would argue, a major success for us to take an iterative approach, which then other people can use and say, these guys have looked at either this data set or this model tested it rigorously. So I know I can trust these results, whether I’m building a product or I’m expanding upon their research for some other effort.
John Gilroy: You were thrown into this world of Python and R and huge data sets and big data and geospatial information and it’s hard to figure out where to start. I think you’re focusing in on tractable results. Is that right?
Ryan Lewis: Yes, that is our intent.
John Gilroy: What are some of the major findings or results you’ve uncovered in the past few years?
Ryan Lewis: We’ve tried to look systematically at all the different aspects of the machine learning workflow from data set requirements to model testing to how you go about labeling. Just as an example, something that’s easy I think to interpret is if you look at the difference between our two SpaceNet challenges.
Ryan Lewis: Our first SpaceNet challenge used a dataset that was a 50 centimeter grown sample distance that’s a type of resolution, mosaic, and then spacing it to use a 30 centimeter GSD or ground sample distance. Simply the difference in resolution alone had a substantial impact on how almost the same type of models performed in identifying building footprints. Now to all of us in the room who are listening, that may sound really obvious and in some ways it is but I think what’s significant about that, is it’s not that we proved out something that is perhaps logically coherent but that we can begin to quantify these results in ways that provide deeper insights into how we want to tune models or how we want to leverage different datasets.
Ryan Lewis: For a specific example, one could argue that it would change drastically between applications, but I think that was a really interesting finding that really jump started a lot of additional work both in SpaceNet and some of our broader CosmiQ applied research projects.
John Gilroy: Earlier this podcast we talked about CosmiQ Works and SpaceNet. Maybe you could define this for our audience. So, complete different organizations, right? Tell us about SpaceNet.
Ryan Lewis: SpaceNet is one of the efforts and arguably it’s the most primary effort that the CosmiQ participates in. Specifically, as I mentioned in the opening, it’s a nonprofit LLC and we run it in collaboration with all the different organizations that you mentioned in the beginning. We started it very informally as a collaboration effort between us and Maxar, at that time it was a Digitalglobe.
Ryan Lewis: The early effort was, going back to that imaging example, was just to see can we get data sets out there, is there a lot of interest in them and if we host a public data science challenge, what type of results will we get. Given the level of interest that we got from just the open source community and the results that we saw, we all collectively agreed there was a “there” there.
Ryan Lewis: It’s something that over the last three years we’ve worked to expand and proud to say that next month or really in early March, I should say, we’ll be launching our sixth SpaceNet challenge. We’re particularly excited about this one because our first five really only focused on electro optical imagery. This is our first multimodal challenge, which will include both high resolution electro optical as well as high resolution synthetic aperture radar.
John Gilroy: Now I think you make it… You’re almost making it fun. The goal is accelerating geospatial machine learning, It seems like a boring class to take in a classroom or something. But if there’s a challenge, different teams across the country working, all of a sudden, innovation comes in and people try to be creative in new ways because they want to beat that other team, state of Washington or California, Florida. Really, it’s almost a gamification of the system or making it fun and really a lot of results come from that, don’t they?
Ryan Lewis: Yes. To your first point, everyone should think that geospatial analytics are fun. If people aren’t doing that on the weekend, that raises concerns on my part, but we’ll talk about that later on. It’s been really surprising and exciting to see the different responses from across the world from all the different challenges. We’ve posted five today, we’ve had over 1,300 responses or submissions on the challenge site that we use and one of our partners’, which is Topcoder.
Ryan Lewis: What I really enjoy is that as we’ve become more mature, we started to have repeat participants across the challenges that look forward to when we release something. They ask us when something new is going to come out. I’m excited to see what they produce when we launch six which is arguably going to be our hardest challenge yet.
John Gilroy: Competition always brings out the best in people, doesn’t it?
Ryan Lewis: It does.
John Gilroy: It really does. Well, if the goal is to accelerate adoption of innovation technologies for the geospatial market, I think you got some notches on your belt to think you’ve accomplished some of that. What do you think is in the future for CosmiQ and SpaceNet in 2020 and beyond?
Ryan Lewis: On the CosmiQ side, I think one of the things that we’ve worked on since the beginning, going back to the strategy part is prototyping what we think the geospatial analytics workflow of the future’s going to look like. There’s a lot of other groups that are working on that or different aspects of it. I think for us, we want to continue to do that.
Ryan Lewis: Specifically as new analytic techniques come online. I think one of the big ones that certainly was getting a lot of interest at ICCV and CVPR in 2019 was something called domained adaptation. I think that has a lot of relevance in the geospatial world. That’s something that we want to explore and benchmark rigorously as we’ve done in different resolution studies and look angle studies in the past.
Ryan Lewis: On the SpaceNet side, our goal is to not only build two different data sets, but launch two different challenges each year. We’re planning for SpaceNet six in March and our goal would then be to hopefully target the launch of SpaceNet seven in the late fall, early winter of this year.
John Gilroy: You’re matching the rapid evolution in technology in ways to handle that, aren’t you?
Ryan Lewis: You bet, and what’s cool is once you have a lot of interest from the commercial and research community, it almost starts driving itself in some ways. For us with the data set alone it’s just been downloaded millions of times. You start getting insights from different end users about what they would like to see next.
John Gilroy: I didn’t realize that’s the number of millions of times people all over the world are interested in trying this challenge?
Ryan Lewis: Millions of times. As I checked last week, in 81 countries in some capacity. So going back to our comment about what people do for fun, apparently there’s a lot of interest in geospatial and in all seriousness, I think it highlights in the classic dichotomy that we see in the analytics world. In one hand it is accelerating immaturity far beyond I think what anyone really predicted in terms of capabilities and prototypes and things like that.
Ryan Lewis: And then the other hand, you have still just a very basic interest in getting access to quality data. For us, we hope that projects like this help feed both those needs, both for the early researcher as well as the venture-backed company that’s building cutting edge products for both public and private sector markets.
John Gilroy: Ryan, unfortunately, we’re running out of time. I’d like to thank our guest, Ryan Lewis, Senior Vice President and Lab Director, In-Q-Tel.
Ryan Lewis: Thanks for having me.
