Transcript: Information and Funding on P25 Phase 2
P25 Phase 2 Roundtable Theme 7, recorded in Scottsdale, AZ, USA. October 19, 2011
The P25 Phase 2 Roundtable was a moderated open discussion. Several themes reoccurred through the day.
This transcript pulls together the phases of the discussion that centered on the theme “Information and Funding on Phase 2”.
Member: So there are issues that we have on this end, things like funding. You know, where are we going to get the funding? And the time that a person is in a position in the communications end of this – again – on the user side can be very limited. And so in most cases, in a PSAP or 911 center, where they are handling the radios, telephone – that type of technology – the majority of the time it is the most junior guy that has been promoted to that position, and he is only there until the next guy gets promoted.
Member: So it is incumbent upon us on the user side to, again, look into the technology, to learn it so that we have an idea of what it is that is out there. And a lot of - as I said before, a lot of us are relying on the standard to be the best practice to be the system design. And that is not what it is.
At this point the roundtable discussion went onto another topic for a time. The conversation then returned to “Information & Funding on Phase 2” as follows:
Moderator: So who should take the initiative, if we agree that it is not a standards issue, who should take the initiative of, say, developing the set of best practices and educating and publishing and promulgating?
Member: SAFECOM tried to start that. If you go on the website, there is some best practices and stuff like that. You know, when you look at the interoperability continuum that we've seen a trillion times, you know, they started on the governance side. They started with SOP’s on the SOP side.
And I think what you are getting at on the technology side, there is the technology that is working on itself. But the actual system management, that might be another component to that. That might be the sixth tier. How do you manage that? What are some things you want in place? What is your recommended software upgrade policy, and how often should you do it? And those are things that everybody in, you know, a modern system, things that you have to do, stuff you didn't really have to worry about 10, 15 years ago. You put it in, closed the door, and never touched it again. Nowadays it is a common problem.
Member: Yeah, I think one of the points is best practices are practices. And they probably need to come out of - it is my own opinion, out of the user end side of the industry more than the vendor side. And maybe APCO is a good home for that, but maybe there are others. But you know, as agencies that have practices in place, and find the ones that work. Put in the effort and capture them, and spread them, to refine them. It seems to feel, that is where the pieces of best practice comes from. I don't think - I think it would be almost a deficit to have the vendor community, the technology side of the community, defining practices for the operational side of the community. I don't think you have Smith & Wesson defining your best practices for how a police officer uses their firearm.
I don't think American LaFrance designs the best practices to how you set up your arrangement of, you know, trucks at a fire scene. And I don't think you have - if you expect the technology side of the industry to drive your best practices and communications - you will have what would look like a Smith & Wesson guide to, you know, an active shooter scenario.
Member: One of the other problems that I think we have is that while the P25 standard has been being developed since, I believe, Craig said 1989, we haven't been using it that long. So we don't have the timeframe to look at it and say, “OK, well, this works really well, this doesn't”, to come up with best practices. We need a little more time utilizing the system to say, okay, this works really well. This doesn't work so well, to come up with this.
Member: There is a first responders database for lessons learned and things of that nature which would be possibly a place to throw a marketplace collective, and people can be submitting things, and they can become live documents. But sometimes when it comes to software - a lot of the people who are writing the software - they want to create it as a service product, and the more they service you, the more revenue they have. So I think that has to be managed, as well.
As an end user, you are going to have to say, “wait a minute, I only want to go through this either annually, biannually, or quarterly at the most”, and it has to be - it can't be - managing a bunch of, what is it, open range chickens. It has got to be a pragmatic approach that doesn't have a detrimental effect on my overall operations, especially if it comes to something as important as interoperability. You have to step back and say, let's slow down a little bit. Unless they are mission-critical software updates, bugs that you found in your software and the bugs you created, I only want to do this when I absolutely have to. And I think that is probably a good place to look at it.
Member: Yes, in the 1973, '74 timeframe, APCO had a project called Project 17 [Project 17 addressed the formation of APCO's Technical Advisory Program], which basically was peers from the public safety community and moved them into a community to help that community develop their plans for technology. That required federal funds. APCO doesn't have that kind of money.
In 1976, the National Association of State Telecommunications directors, now State Telecommunications Professionals, did the same thing with telephone systems. And both of those worked very well, but both of those tend to exclude on a high level consultants like Neil. They both excluded industry from participation. There has to be a process put into place.
Number (1), you have to depend on somebody to fund it: which would mean the federal government or a lot of rich manufacturers or somebody. And number (2) that would allow industry to participate on a peer basis. That would allow the public safety community, which is also an important part, to participate and would allow all of those political national associations to participate, and you would need to find, develop best practices. You need to have people who had the experience like… I don't mean to hit you John. You will get real Willie Huff after me. Willie is his commissioner.
You need to have people that have that practical experience in the process. You need to find a way for them to integrate into that process without taking away from their own organizational responsibility.
So ‘best practices’ is not a simple thing. The ERC [Emergency Response Council] tries to do it on a voluntary basis by sending people to meetings through SAFECOM OEC [Office of Emergency Communications]. And it works fairly well, but what is happening is the Feds are developing the intellectual property, and the users are basically saying, yeah, that -- in my mind, that is not good. Most of those feds don't have a clue of what you guys go through, and to just accept that is not right. You need to have your input into that process. It is one person's opinion.
Moderator: It is Phase 2, which nobody has got any prior experience, going to be an opportunity to start this, or is it going to be a hindrance, or is it going to be an opportunity lost because there are now more choices out there? There is not just Phase 2. There is DMR, NXDN. Is this all just going to evaporate into nothing?
Member: Technology should be transparent to the problem. The problem exists regardless of what technology he or she chooses to use. The problem is, you have an unfended mandate, you have a drop dead date, you have people in a poor economic condition without economic resources, without intellectual resources, without political resources to fix it. So that is your problem.
At this point the roundtable discussion went onto another topic for a time. The conversation then returned to “Information & Funding on Phase 2” as follows:
As we look at the mandate, forget everything else, in the marketplace, we have a lot of manufacturers that are dumping a lot of money into, quote, Phase 2 technology, end quote. That money is going to need to be recouped from someplace.
So they are looking forward to some form of increase in the market place. I'm confident of that. Although I can't speak for them, because I don't build products, but I'm confident they are looking for a return on their investment. So there has to be some incentives built into the process to take care of that, but the incentives we are focused on is the incentive for the public safety community.
Member: And how we make that a win-win for public safety an industry. I don't know that…I haven't looked at…I guess, it is Steve's regional coordinator that you are referring to?
Member: I haven't looked at what they are proposing. I suspect you are absolutely right. Some people will like it. Some people will think it stinks, and I think that is appropriate. I think that kind of discussion should go on in an open forum so we can arrive at some conclusions. And so that industry knows where they are going.
At this point the roundtable discussion went onto another topic for a time. The conversation then returned to “Information & Funding on Phase 2” as follows:
Member: I think the elasticity point is one of the things that I think is driving some of the changes that I think are going to be placed in the P25 and TIA: trying to make those standards more plain-Jane. Because they have reached a point in elasticity in the economic sense: the price point is higher than many people can provide. And that is in a marketplace where there are multiple manufacturers providing multiple tiers of products.
But we have to reach beyond that. We have to reach beyond making sure that we understand, we are part of the global context of the world.
Member: I think to echo what Steven is saying, one of the few things that is actually advantageous to the migration, and we found exactly the same thing you are speaking to, it does help from a funding perspective because of how it changed in the last two years in particular.
I think we are finding there are no longer these huge grants unless you live in one of the UASI areas that homeland security is being at most risk. You found yourself in a position where you are no longer looking for a big grant to fund your forklift upgrade. So you are put in a position where you are trying some self-funding. You are trying for grants in multiple ways hoping to cobble together your bogy, if you will, for the worst case scenario on expense. Where this has helped is in this mode of progression where you do have to start somewhere, it lets me take it off in much smaller bites. And this was very beneficial in approaching my council, because having a migration plan in place such as this, it recognizes the economic constraints. It allows us to creep forward as opposed to run forward, and we are definitely expecting to have hard moments during that. We are modeling everything we can possibly get our hands on. I think to your point, as well, I think we are going to end up overbuilding a bit from a traditional standpoint.
Moderator: You have to have a certain amount of trust, but nobody has tried this before. Isn't that right? And that has to be part of the plan.
Vince, what would be your advice on, you know, how to start the ball rolling in this complex multilevel migration exercise, because the funding seems to be, you know, a key part of this? What can I do in order to begin the process, and whatever it is I do, I need money for it.
Member: I think it starts with a couple of things. First being, having an understanding of the difference between a pass-through program and direct program. Direct, obviously, come from grantor to grantee, basically, no middleman. A lot different from a pass-through process where, generally, funds originate, say, in this case with DHS, FEMA, pass through the state administrative agent to support local level projects.
So understanding the mechanism by which that money is made available really significantly impacts your next step for building awareness of the need for this particular change. Unfortunately, P6[Funding Program 6] of the world aren't really in existence right now. Will that ever be reconstituted? Maybe yes, maybe no.
From the beginning, though, when I would develop or if I was developing a project, basically taking a step back and understanding what you need to build off in order to get to the point where you need to be with communications interoperability. And some programs, the state of Homeland Security program, the Emergency Management performance grants are very network friendly, are very IT backbone friendly, and I would really use those type of programs to set up the network and use other maybe smaller programs for the build-off over time.
We are dealing with very fluid grant landscape programs that have been around in the past, not around now. Some sustainable programs that are still around aren't receiving the appropriations they have been in the past. You mentioned UASI [Urban Areas Security Initiative – a funding program within the umbrella Homeland Security Grant Program] is a perfect example. They went from 64 urban areas in 2010 down to just over 30 in 2011.
So, really, solidifying your grant-seeking strategy has to do, again, with building an awareness of what it is you are trying to do. It starts, obviously, with the local level, ties in with playing the game, working with some of decision-makers, with your regional committees, some task forces that might be present, and, obviously, working with your state administrative agencies. But leveraging both geopolitical factors as well as geographic factors, and a lot of the rural communities might have at their disposal U.S. Department of Agricultural programs, for example, in the same way that a metropolitan area might have (inaudible) or MMRS [Metropolitan Medical Response System – a funding program within the umbrella Homeland Security Grant Program] or maybe some other funding streams. So, basically, getting the ball rolling, and really, from the beginning understanding what you are trying to do, because a full, you know, statewide project, it will look at lot different from a local community or a local county in what they are trying to do. So it is defining the variables early on.
Moderator: Can I summarize that a little bit? So the idea is that if the big monolithic grants are not there. But there are a lot of little smaller pots that you can be advised of, provided you're clear about your project from your communications point of view wants to achieve in various stages. And then you get an advisor to tell you all of the different little pots that you can touch there based on what you want to do, where you are, and who you are working with, not just now, but maybe three years from now. So there is a way of getting going. But it is a trickier landscape, because it is more like Florida. There are a lot of little bayous there, there are a lot of little crocodiles, a little islands.
And I suppose part of the complexity, is that you still have to, as Frank brought up, maintain communications here and now, and the migration has to preserve that communications that everybody depends on all at the same time as rebuilding the ship.
Member: Mr. Chairman, I don't think you can ever lose sight of your vision of where you want to be. I mean, we are talking about near term, but you always have to have a focus on where I want to be in ten years from now. We discussed at breakfast, and somebody said, well, I think it was Neil. In fact, somebody said, most public safety agencies really don't know what they want to be. They just know what the last vendor in the door basically is selling. That is probably true, but the fact of the matter is, we, people who have been in leadership positions in public safety, have not done a good enough job mapping out where the future should be. And the whole discussion of Phase 2 shouldn't be that difficult if we've done a good job mapping out. That is our problem. We failed.
Moderator: That is why we are having a round table. We are filling the vacuum.
Member: We are the guys that failed. We shouldn't have to do that, but we do, and we don't have a forum. We don't have a mechanism to let the public safety community be informed about what they are really buying as opposed to what they think they are buying, whatever.
Member: What the manufacturer says they are going to have.
Moderator: And there is no certainty. Nothing that you can say, well, you know, I hear what you say, but is it actually secured by a standard testing…
Member: You are part of the equation. Industry is very reluctant to have these kind of discussions out in the open. I mean, I've hounded Neil with PTIG about them. We need to bring some of this controversy out in the open, not just for industry, but for the public safety consumers until the industry can see their perception is not necessarily correct.
Moderator: And also -
Member: What -
Member: What some manufacturers do, they will sell you a product that says the device is ‘capable.’ With not really understanding, you think you've got something that will do something, but oh, there is an additional fee to turn it on to make it work. And that really ticks me off. Either you are buying something that will work or it won't work. So that is really deceptive marketing in my opinion.
You know, either you have a feature on the radio that will work when you purchase it, you understand it will work, not some weasel words that it is capable of working but you have to pay an additional fee to turn it on. You know exactly what I'm talking about.
Moderator: Is this just something about the education of the end user?
Member: Yes, it is an education for me. And I know to ask those specific pointed questions when negotiating contracts.
Member: One of the things about, you know, buying into these systems, if it is a new person, new government agency, and so on, I guess, by comparison, I going into this, because of the amount of support, planning, everything else that goes into it. It would be like turning your vehicle fleet into natural gas vehicles. It requires all new infrastructure. You may only be able to support it within 50 miles of your jurisdiction. It is incapable of going to the state capitol, or whatever, because there isn't a pumping station that you can hook it up to because you have a unique connector and that type of thing. It all requires a lot of heavy-duty planning and support as opposed to just getting a bunch of gas vehicles back into the fleet that you can replace those as needed.
Member: One of the problems with that is most of the local small government agencies don't know what they need.
They don't have a clue what it is they are looking for, and they put their trust in their local radio shops.
Member: And depending on what the local radio shop is trying to sell today, is kind of what they get.
Member: What they are being incentivized to sell.
Member: A lot of the smaller jurisdictions, because the people that are being charged with doing the technology acquisition aren't necessarily technologists and don't understand the technology. They think P25 becomes this thing that is going to protect them, that that is their Kevlar vest. And boy, it can't be further from that, because it gets tweaked and turned around and value engineered, and, “oh, you mean you wanted that," after the thing has been deployed. And, by the way, that is a change order.
You know, it gets to be a type of…the sins of the sale, because of the buyer doesn't know exactly how to buy it. The seller has a propensity to sell it, because he needs to get his little bonus bogy, and then you end up with something that becomes very convoluted.
The problem is, P25 - the education of P25 down at the buyer level - needs to be articulated much, much more. And I think that is not only done by the vendors, but it is needed to be done by the association.
Member: We seen it earlier today. Somebody mentioned due diligence. I see in some agencies, particularly the smaller ones that aren't as technical, they try to use the standard, you know, as a stamp to replace doing due diligence.
Member: And, you know, it just doesn't work. The standard was never going to be to the point, and I can't imagine it. I kind of use analogies for other markets. I can't think of anything that you buy the standard stamp replaces your due diligence as a buyer, yet we see it happen in our market over and over again.
Moderator: Is there some mitigation that can be done? For example, people see what a clean RFP looks like that, you know, minimizes some of the risks, and that is part of the education. Is that a useful ploy? I mean, how do we educate people at the buyer level?
Member: What you just mentioned is out there already. Most of the folks don't know how to go find it. And a lot of the people, the consultant houses, for instance, they have various levels of how complex the RFP becomes. Just recently, your one consulting firm had a 250-page RFP. I mean, it got so far down into the weeds, that you were picking the pepper out of the salt. But you don't necessarily have to do that. The biggest thing is the due diligence, that needs analysis, identify the gap analysis, review the technologies that can get you there, figure out what your actual economic situation is, so you can match the technology road map with the economic condition. Then you have a formula for success. That needs analysis is so critical before you do anything else. You have to have that needs analysis.
Member: Totally agree.
Member: You will see on the APCO has a discussion board, and you will see an agency request, hey, does anybody have an RFP 4XYZ?
Member: I usually shoot off an e-mail. Kind of like generic RFPs will miss a lot of the course. Smaller agencies will do that even if they moved away from trusting their vendor completely. They will pick up somebody else's RFP that happened to work for them and put it out expecting it to work for that agency. And it may be chance, and it may not.
Member: That is a dangerous proposition.
Member: Yes. So it is an education problem at all levels in the industry, because we are so fragmented in public safety, it makes it a public challenge.
Member: And we also have on our own, still with the same part of doing the due diligence, it is the folks that respond to that RFP that you have to look at them and put your trust in it and when we - when our RFP was extremely brief for our system, and we had to respond to it. And we had to literally sit down and go through, both to try to do a justification, to the point where we went to actual two cities that were using one of the vendors to look at the system, because we weren't familiar with the company.
Again, we did our due diligence in doing the research on everything that was proposed in the one proposal, because the other proposal, everybody was familiar with that particular company. And so, again, we had to prove, because not only was there a cost difference in the two proposals, but we wanted to be able to prove beyond a question of the money that, yes, this system is going to work. It is what we need. You have to have that trust in who it is that is coming in the room to show you a system.
Member: A tremendous amount of money wasted all the time in public safety by agencies, all over, reinventing the wheel with these RFPs because lack of standards that should be able to define how a system is going to perform. Technically, it should be definable, as far as the fact we want this radio to work with this signal level, through this typical type of build infrastructure. Number (2) the best practices to back it up and supporting and maintaining that to the point where that could even go to a standard itself. But it isn't there.
Member: There is one actually, it is clothed in a VHF trunking deal, but there is one out there that is a fairly good boilerplate that could be used, and you could tweak it for UHF or 800MHz, but I can forward it to you, if you like.
Member: Well, again, I'm talking about…I'm referring to a standard…
Member: That could be called upon that could eliminate a lot of the creative language, a lot of the need for an agency, jurisdiction, or whatever, to sit there and hire a consultant, to draw up the maps and call out of all of the specifications.
Member: Maybe we should get a book of Radio Systems for Dummies.
Moderator: Maybe we could come to that during lunchtime when that occurs.
Member: But the creative language I want to ask the vendor community, the creative language put in the RFPs is usually motivated or pushed in through the vendor side in order to stack it in one direction. And I often see so many are available or on a Project 25, quote, unquote, ‘standard’. However, the way it is graded, and the way some other creative language is in there, creates a sole source contract.
Yet I don't see the vendor community in uproar. I don't see one vendor saying, that is hanky. You can't do that. That is not right. What they do is, I lost that deal, and they go on to the next one.
Member: How many TETRA systems around the world are like that?
Moderator: I think we are getting off the topic right now. That is an interesting point, but I don't think it is one we can actually look at.
Member: Going back now to the scope of P25 and P25 standards, going back to what you asked earlier on, if these things should do something that is regulated. I think the biggest problem that concerns P25 standards is manufacturers including features in a product that are not part of P25 standard and locking the consumer into using their product by offering this feature for free, or, hey, you can use this. This is a nice feature, and now it’s the only manufacturer that offers this feature.
Member: I think that is what needs to be regulated to very simply ask for disclosure of any features that are not P25.
So the product follows P25 standards, but if there is a feature I'm selling you that is a proprietary feature, not part of P25, I should be required to disclose that to the end user.
Moderator: Shouldn't the burden of that fall on the buyer, due diligence?
Member: The procurement.
Moderator: Show me all of the features and point to where they are in the specification. Isn't that one move?
Member: Phoenix started out - the Phoenix system, Phoenix/Mesa started out as a P25 standards based system. They invited all subscriber people that - manufacture subscribers, so on and so forth. Once the system was in, as things progressed down the road, it became pretty much a sole source, only because some of these creature features were adopted and that just locked it. It really is uniqueness, or whatever. It moved away from what should have been a standards-based system at a certain point because…
Member: Then it is no longer a P25 system by definition.
Member: Right. Yeah, exactly.
Member: That is debatable in the industry and TIA. The core standard is still there, so you can't say it is not a P25 system. The issue really is, it is a buyer decision to buy those proprietary features…
Member: I agree with you; however, as soon as you drive an automobile on propane, it is still an automobile, but it is not running off the standard gasoline.
Member: That is right, but - and this is three years ago probably, I proposed that we require that the P25 core systems, anything outside of that, not be called ‘P25.’ And that failed. So I said, okay, well, how about if we agree that when you bid a product, you will list - you will provide a checklist that identifies all of those components that are P25 compliant and all of those that are not, just as a standard part of the process. One of the majors agreed to it. Nobody else did.
So it is a business decision. And in the end, when you look at that business decision, they are covering their tail. They want to make money, the salesman.
Member: So it’s ‘Buyer beware’.
Member: You are going to ensure you are going to buy from him. The burden ends up back on the public safety people.
Member: You have to hire consultants like Neil and others in this room to help you wade through that, and that only guarantees it from day one, because after day one, it is all a new ball game.
Moderator: Question from Robert Simmons, I think.
Member: I was going to try and go back toward migration, and some of the concerns that I had. These are more technical. I don't know if you want to continue with that.
Moderator: Would you be interested in following up this particular line or…?
Member: No. I agree with what everybody has been saying. There is a lot of ‘buyer beware’ out there. I agree with Craig. There needs to be something out there that kind of marks, if a certain manufacturer markets a certain product or feature, they need to say it is based on TIA 102 BACA or whatever.
Moderator: We discussed at lunchtime… just ask this first, Craig, and then I will get on to you, about leasing. Why don't they like leasing?
Member: I think that reliance there is on redundancy that is built into a cell phone system, based on the needs of… the persons that use it. Where the needs built on a public safety system are based off of the needs because, let's face it, when we need to use it the most, your system may not operate. So the redundancies are different to build off of. As long as those needs are being met with the infrastructure side of it, then I agree. I mean, a…
Member: A lot of the states' purchasing laws may prohibit a leasing contract. So you have to look at that aspect as far as from a leasing perspective.
Moderator: Is anyone aware that there are purchasing arrangements that might require a lifetime greater than three years, actually require a stipulated lifetime of a minimum of, let's say, 7 years or 10 years? I mean, I've seen at least one RFP that did that.
Member: I mean, there are fees that require the product is supported for…
Member: I think you run into a couple of problems. First of all, I gave a speech in 1980 at an ICA conference, which most of you probably don't even know what it is, saying that at some point in our lifetime, my lifetime, public safety people will lease their equipment and not buy it.
Having said that, and I think you get into a real problem when you talk about throwaway equipment, because most policemen, most firemen, if a knob falls off the radio, it is the whole system's fault. They are not very tolerant. You are talking about a massive culture change here.
My old handheld yellow radio that you buy at WalMart to something that you buy commercially. You talk about a lease system, you can get away with that, and it will work very well as long as your contract. Going back to what Neil said earlier, as long as you have a contract that will accommodate that.
The state of South Carolina used to lease their system from Scana. They now lease it from another vendor or radio vendor. State of Georgia looked at leasing a system from Scana. There are a number of states out there that actually lease systems. So the concept is not foreign. What is foreign is being able to sell it. Being able to sell it, being able to provide a radio that is going to meet that public safety standard. Why we pay so damn much is we ask for so much. If we don't ask for so much, we won't pay for that.
Member: Part of the issue, too, is, again, on a state level doing a lease for a statewide radio system, in New England, we have so many little fiefdoms that they are reluctant to tender into a lease, because then that is going to taking away from their budgetary powers. If we are buying -- if we buy it outright, then that is a one-time deal. They bond it and pay it back and…good-to-go. Now you are in a lease. That means, okay, well, a new election comes up, and this next group comes into office now stuck with this lease program. And they are reluctant to get involved in long-term payments, because if the tax base has to go up, if something happens with the economy, it reduces their budgetary powers.
Moderator: I will take one more question, and then we will probably switch topics again.
Member: Also, a lot of manufacturers don't want to agree to the terms. Like in the state of Mississippi, they have the right to terminate a contract, a lease contract, at any time based on funding. And that… and they won't sign anything that doesn't have that stipulation in there. So, yeah, we will do a lease.
And we have a lot of tower leases that have that clause in there, but if, you know, the state wanted to terminate that lease, they can do it at any time.
Moderator: So what does the average sort of police or fire agency do in order to discover what their security needs are? Where do they make a start. I mean, again, a decade ago, it might not have been an issue, but now all of these things have turned into issues. Funding, security, third-party packages, data applications. Where do you go? Anybody?
Member: The needs analysis of the project really to understand what is needed and what is not needed, what the value is, and then, you know, have that traced through the project. I think that kind of ties back to what we talked about earlier with, you know, cut and paste RFPs. They may be very good, but they are not what your agency needs. And that agency may need, along with the radio system, their IT group may need to know how to do security audits, and they may be captured in the needs analysis that is specific for that agency.
Moderator: Do you think that users can do an awful lot for themselves irrespective of contacting vendors and consultants in terms of that needs analysis?
Member: Depends on the agency, and we had a side conversation about this. Police departments buy cars all the time. The guy who buys the car is in that position until he leaves, may buy 7 to 20 sets. You buy radio systems fairly infrequently. The chance of somebody being in a position in anything but the largest department to actually have built any real experience is thin. And that points to the needs for, you know, not to self-advertise, but…
Member: But you are.
Member: But the need for consultants in the industry to shortcut that piece. If you are large enough that you have a department that manages it, that is great. But for most departments, you are too far down the learning curve to really do the job that your department needs done. Not that you are doing a bad job, but the technology is moving very fast.
Moderator: Well, I mean, you can do something, surely. You know what you have. You know where it is deployed. You know how your department works. You know how your communications need to go. You can certainly make some important basic first steps in terms of your expertise rather than some external party's expertise. Wouldn't that be right?
Member: It is probably better than nothing. The best is where the project is run as a project team, and you pull your operational expertise from inside the group, and you pull the market expertise from usually the outside. Most departments aren't big enough. But not turn it over to a consultant and say, “Buy me what I need”. It really needs to be a team, and the agency needs to understand that there is a lot of work on their side, too.
Moderator: Isn't security more than just worried about bad guys and worrying about inadvertent mistakes? You brought up cloning, for example. Even the people who provide support, knowing what you got out in the field, isn't security actually, you know, a big topic that on the one hand covers, you know, malicious intrusive things, but it also covers protection of the information, the data you've got there, making sure that everything runs. Doesn't that have to be part of it? How do people deal with that now? Asset management must be part of it?
So it is learning a whole new lot of things to worry about, like system keys, like asset management, like authentication, which you brought up. How do people get advice on how to do that? Vince.
Member: Talking about conventional systems, I hate to say it, but we kind of learn as we go, because they are so new. Things that haven't existed before as concerns now are big concerns. I will give you an example. Traditional radio system. You have a repeater, you have phone lines tying it all together. There was a voting system. So traditionally we have a repeater, we have some receivers, and some phone lines tying all the sites together, and all you have to worry about is that site is locked. That is about as far as security goes in the radio system.
Now, a conventional system that has an IP backbone, that has all of these nice features, introduces a whole new set of rules that need to be followed, and concerns that need to be addressed. For example, we have, you mentioned, wireless broadband network. We need to make sure that is encrypted, because data is sent back. Not just that somebody can just listen in. Can somebody log in and mess with the network? Site access is no longer if somebody can go to the site and unplug the repeater. Now you can just plug into the network and get access to the network. We do encryption. Where does the encryption end? It has got to end somewhere. So now that is a concern. Now we have a future of being able to remotely log into the system for service. Who has access to that? Who has…okay it is encrypted. Who has a password for remote access? The radio shop. Well, who in the radio shop? Can somebody at radio shop acquire the password and get into the system? That is a concern.
Now we have the ability to be notified if there is a problem with the system through e-mail. Okay. Who gets notified? Radio shop. What if the radio shop doesn't respond. What is the backup plan? So there is a lot of things that now we have to consider and worry about that really never even existed before. So it is all this new features and benefit that we've introduced. But along with those benefits and features, we bring in new concerns that have to be addressed.
Moderator: That have complicated the whole decision-making process. Is that right?
Member: Yes, complicates. Well, see, it kind of falls back to the vendor. Now integrating the system to go over all of this with the client and address all of these issues. It is very difficult. I would say it is impossible for the end user to be aware of all of these ahead, because the system is so new being deployed.
These issues are just being kind of discovered on-the-go as the new features are being introduced. So…
Moderator: But at the same time, we want to make, you know, it possible for people to actually buy into these features now, which implies, in probability, some form of yardstick, like CAP testing, the ability to understand the requirements in the vendor-neutral way so people don't receive their primary education from the vendor. Would that be right?
Member: Yes. That would be great.
Member: There are four definitions for rules, responsibilities, and skill sets that are required to get one of these systems going. A lot of people that have bought into these systems wind up learning all about the things that he was describing and then all of the other training and whether or not you even have the staff that can understand any of this stuff they are going be force-fed to be able to set up the program. Heck, most of them had problems just programming the conventional radios with the latest software, much less the complexity of the latest greatest subscriber software, along with all the system infrastructure.
Moderator: So if we were trying to work out the sort of general game plan for making it easy and minimizing the risk as much as possible for somebody to move from whatever they have now to Phase 2, what would be the big buckets?
Member: If you had sat through network management classes and certification, by the time you get out of there, you are cross-eyed, and your brain is tied up in knots. First of all, it needs to be more straightforward. All the crunching needs to be done on the back side of a GUI, in my opinion. Because what they have done with the way they do the fleet mapping, it is a very complex, especially when you get into regional systems and multiple system keys, and the fleet mapping that has to go on on that. And that management system, and the ease of managing that would probably be a real big differentiator if there can be some type of standards based management system that allows you to take care of that in a more straightforward way. Because right now, it is Chinese trigonometry, let me tell you.
Moderator: How about estimating the complexity or the cost of change? You know, what aids are there to helping somebody work out, well, when can I do this? How much might I require? How long is it going to take?
Member: I would say the consultants are probably the only ones that can come in and start putting those pieces together. There really isn't anything you can draw on.
Moderator: How do the consultants learn?
Member: Either experience or go back and interview.
Member: I would just say it is a bridge you have to cross before you even can have the gray conversations when you are working with the decision-makers and the people steering the available grant money. They want to know you've done your due diligence, you know what your system costs, you know the life cycle of what you are getting into, you know that it fits well within the performance period of that particular program. So get your building materials and your budget in place as much as possible before you have those conversations, because that has to be down on paper before anybody can advocate for your project’s inclusion in the local and state level plans. So how you get to that point, I think vendors are the subject matter experts that make the most sense.
But it is certainly out of a collaborative approach that one can get those variables defined.
Member: I would just…vendors hide so much from what heavy lifting is actually going be required by the customer to get the deal done. That may be a cold hard statement, but there is so much hidden - where it may be a paragraph or two - but it involves a body, an entire skill set that has to be created for an individual to be able to support something. I don't think vendors in any way, shape, or form have ever been forthcoming and really defining what it is going to take to build one of these.
Member: I have one question for our consultants. How many of you have ever worked for a vendor?
Member: I did.
Member: Have worked for?
Member: Individually, yes.
Member: A lot of the technology where this comes up and training about is from OJT [on-the-job training], I guess, put it that way. And for whatever reasons we decide, like my case, I decided to go play some cop and come back into the field, but this time as a communications guy. So the vendors sometimes have the best resources because they are not teaching school anymore.
So where do you go? We almost have to rely on the vendors, but we look at it with a skeptical eye sometimes because we also know that you are trying to do something.
Member: There are more than one vendor. That is where it may be a little harder for an agency to do on their own. They usually have their existing vendor or one that is coming in to sell.
A consultant is able to go to more than one vendor, maybe or maybe not disclose who the client is, depending on the requirements, and also has a history. If we get a rough order magnitude from vendor Z, trying to stay away from M's an H's, but…and then we look at an actual bid that came in six months earlier and say, this doesn't line up.
They are padding this or padding that.
Member: Vendors are real good for being able to provide the training, the knowledge base for learning something. But they absolutely back away from helping you define, what is this going to take for you as an agency, the amount of staffing, again the skill sets for what it is going to take to be able to bring one of these systems on line. No question, the M's and so on and so forth out there, they have the entire knowledge base in how to make their system get up and rock. But they aren't the ones that will ever really define any and all of these bodies of people to be able to support this thing.
Member: If I may interrupt. In a couple systems I'm familiar with, they still have service contracts with the vendor to maintain the backbone and the sites. I mean, and the consoles, the only thing the radio shops are taking care of are subs.
Member: Some of the systems I set up, essentially, it comes down to the agency will have to have a NOC, a network operation center, essentially where they staff that alarm system 24 hours a day, and they need the support staff to support that group. So it gets bigger the bigger the system is.
Member: It wasn't in the proposal.
Member: It says under their customer responsibility; right?
Moderator: Craig wants to say something.
Member: Historically, the steering committee…I personally never ever told anybody going to Project 25 is cheaper, because it isn't. The point you are making, you have to have dedicated people, one or more, depending on the system size. It is absolutely correct. You can't get around that. There are people, in many cases, they are doing something else now, and you are going to have to replace them.
Number (2), LA County is not an anomaly. Many of the large users today are tied to the supplier of their equipment. Start off, I will do it for one year. Then they go for two years, and then they go for five years. And the problem you run into is the same problem I ran into in the telephone business with the state. I could never afford to provide the level of training that AT&T could provide. I could never afford to provide the parts and maintenance that AT&T could provide. And that is kind of the box we get into with technology. There is only so much we can do, and it is not because we don't want to, it is because we don't have the intellectual and capital resources to do it.
Member: So I think that any consultant worth his salt - his or her salt - is going to tell somebody right up front, “this is going to cost you more money”. I know when we've done that, it has been fluffed off as something stupid to say. But the reality of it is, it is a totally different world than the world I grew up in where you take a base station on the mountain, hook up an antenna, and away you go.
Moderator: We are introducing a new technology such as Phase 2. Are consultants not in the same boat, too? Trying to get, you know, estimates of complexity, time, effort and so forth, that are going to be useful to their clients.
How do you educate them? How do they educate themselves?
Member: I personally don't know of any consultant that would go into an organization and say “Guess what, kids, this is more expensive. Here are the problems you are going to be facing”. I don't think they would try and interject their knowledge for the person's knowledge who they are dealing with and say “it is going to take this much time and it is going to take this level of education”. I don't think they would do that, but I do think they would set up the warning flags, and together as a team, they would work through what has to go into place to accommodate their needs. Some of this training that Neil is talking about - some of this stuff has to go on long before the system even comes on line. And guess what? By the time it comes on line, some of it is stale, so they have to send them back. You deal with one major manufacturer. They come up with an upgrade every 18 months, and the software, well, that shouldn't be a big deal, but guess what, it is. And everybody that has to deal with that software has to be trained to deal with it, or they have to have a contract with them to take care of the work. And that is a reality. I mean, I don't know whether it is good or bad, but that is a reality of the technology.
Member: And the flaws in the software is, the manufacturers give buggy software, and so they are forcing all these software revisions on you. Because, by the way, we found this bug. Don't worry about it. Doesn't cost you anything. Here, put this new firmware. It takes 200 man-hours to put the new software in. It costs money, and it is also an opportunity to screw up the security of the communication system. So the software needs to be debugged before it goes out, first of all, and they don't do a good job of that at all.
This document is the seventh transcript in a series of theme-focused videos of the P25 Phase 2 discussion.