We started as IT auditors. Before we ever managed a single device, we spent years reviewing managed service providers on behalf of their clients. We’d sit down with the contract, the invoices, the actual tool deployments, and the support history. Then we’d compare what the client was paying for against what they were actually getting.
The gap was usually significant.
This checklist is the framework we developed through those audits. We use it to evaluate other providers, and we hold ourselves to the same standard. You can use it whether you’re shopping for a new MSP, checking up on your current one, or trying to figure out why your IT still feels broken even though you’re paying someone to manage it.
The 10-Point MSP Evaluation Framework
Each point is scored on a simple pass/fail basis. An MSP either does these things or they don’t. There’s no partial credit for good intentions.
1. Monitoring Coverage
What to Check
Ask your MSP: which devices are being actively monitored right now? Get the list. Compare it to your actual inventory.
What We Found in Audits
In roughly 40% of the contracts we audited, the monitoring agent was missing from at least one device that was supposed to be covered. Sometimes it was a laptop that got reimaged and nobody reinstalled the agent. Sometimes it was a new machine that was set up without notifying the MSP. Either way, the client was paying for coverage they weren’t receiving.
Monitoring coverage is the foundation of managed IT services, and it is the first thing that breaks down when a provider gets sloppy. A monitoring agent should be installed on every device covered by your agreement, and the provider should have an automated process for detecting when an agent goes offline or gets removed. Ask your MSP for a current device list with agent status and compare it to your own hardware inventory. If any covered device is missing an agent, that device is unmonitored. It is not being patched, its security status is unknown, and problems on that machine will go undetected until something fails visibly. A good MSP runs this check automatically every day. A bad one only checks when you ask. The gap between those two approaches is the difference between proactive IT management and expensive neglect.
Pass/Fail
Pass: Every device in scope has an active monitoring agent. The MSP can prove it with a current report.
Fail: Any covered device is missing an agent, or the MSP can’t produce a current device status report on request.
2. Patch Compliance
What to Check
Ask for a patch compliance report. What percentage of your devices are running current operating system and application patches?
What We Found
Patching is the single most neglected service in managed IT. We audited providers who claimed they patched monthly but had devices 60 to 90 days behind on Windows updates. Third-party application patching (browsers, PDF readers, Java, etc.) was even worse. Many providers only patched the operating system and ignored everything else.
Pass/Fail
Pass: 95% or higher OS patch compliance within 30 days of release. Third-party applications are also patched on a defined schedule.
Fail: Any device more than 45 days behind on OS patches, or no third-party patching program in place.
3. Backup Verification
What to Check
Ask your MSP: when was the last time you tested a restore from our backup? Not when was the last backup taken. When was the last restore tested?
What We Found
This was the most consistent failure across every audit we conducted. Backups were configured and running, but nobody ever tested whether the data could actually be restored. We found backup jobs that had been silently failing for months. We found backup retention policies that kept only 7 days of data, meaning a problem discovered on day 8 was unrecoverable. We found providers who could not produce a restore test log because they had never performed one.
Backup verification is the difference between having a backup system and having recoverable data. Running a nightly backup job is step one. Verifying that the backed-up data can actually be restored to a working state is step two. Many MSPs complete step one and skip step two entirely. Ask your provider when the last restore test was performed and request the documentation. A verified restore means someone pulled data from the backup, restored it to a test environment, and confirmed the files were intact and functional. If your provider has never done this, your backup is a hope, not a plan. A good MSP performs verified restores on a scheduled basis, at least quarterly, and documents every test with timestamps, data volumes, and results.
Pass/Fail
Pass: Verified restore test performed at least quarterly. Documentation available showing date, data restored, and result.
Fail: No restore test documentation, or the last test was more than 6 months ago.
4. Security Tool Deployment
What to Check
What security tools are installed on your devices? Are they just installed, or are they actively monitored?
What We Found
A common pattern: the MSP installs an antivirus or EDR tool during onboarding, then never looks at it again. The tool generates alerts. Nobody reviews them. The tool’s license expires. Nobody renews it. The tool gets disabled by a user. Nobody notices. Having security software installed is not the same as having security. Someone has to be watching the alerts, responding to threats, and verifying the tools are running.
Pass/Fail
Pass: Security tools are installed on all managed devices, licenses are current, and the MSP can show alert review logs from the past 30 days.
Fail: Security tools are missing from any managed device, licenses have lapsed, or the MSP cannot demonstrate active alert monitoring.
5. Helpdesk Performance
What to Check
Pull your support ticket history for the last 90 days. Look at response times, resolution times, and open ticket count.
What We Found
We found providers with average response times of 12+ hours on tickets that should have been addressed in 4. We found tickets that sat open for 30+ days with no updates. We found “resolved” tickets where the resolution was “user did not respond” because the technician sent one email and gave up.
Pass/Fail
Pass: Average response time meets SLA commitments. No tickets older than 5 business days without a status update. Resolution rate above 90% within SLA timeframes.
Fail: SLA response times routinely missed. Open tickets with no updates for more than a week. Pattern of closing tickets as “no response” without follow-up.
6. Documentation Quality
What to Check
Ask your MSP to show you their documentation of your network. Passwords, configurations, network diagrams, vendor contacts, license keys.
What We Found
Documentation is the thing every MSP says they maintain and most don’t. We found providers who could not produce a current network diagram. We found password vaults with entries dated three years ago and no updates since. We found no record of vendor account credentials, meaning if the MSP relationship ended, the client would have to call every vendor and start from scratch.
IT documentation quality reveals how seriously a managed service provider takes their operational standards. Complete documentation should include a current network diagram, an inventory of all managed devices with serial numbers and warranty dates, a password vault with credentials for every system and vendor account, configuration records for firewalls and routers and switches, license keys and renewal dates for all software, and contact information for every third-party vendor the provider interacts with on your behalf. This documentation should be updated every time a change is made. If your MSP cannot produce a current, organized version of this information on request, that is a significant operational risk. It means a transition to a new provider would be chaotic, an emergency involving a system nobody remembers configuring would take longer to resolve, and your business continuity depends on the institutional memory of individual technicians rather than documented procedures.
Pass/Fail
Pass: Complete, current documentation including network diagram, password vault, device inventory, vendor contacts, and license tracking. Updated within the last 90 days.
Fail: Missing, outdated, or incomplete documentation. Anything more than 6 months out of date.
7. Reporting Consistency
What to Check
Have you received a report from your MSP in the last 90 days? Was it useful? Did it contain actual data about your environment?
What We Found
Some providers send reports religiously. Some send nothing. Some send generic PDF reports generated by their monitoring platform that contain 47 pages of graphs nobody reads. A useful report is one that a business owner can read in 5 minutes and understand what happened, what’s healthy, and what needs attention.
Pass/Fail
Pass: Consistent reporting on a defined schedule (monthly or quarterly). Reports contain device health, patch status, security events, backup status, and ticket summary. Readable by a non-technical person.
Fail: No reports received, or reports are generic vendor dashboards with no business context.
8. Vendor Management
What to Check
When your internet goes down, who calls the ISP? When your printer lease is up for renewal, who handles it? When your phone system has an issue, who coordinates the fix?
Pass/Fail
Pass: MSP acts as the single point of contact for all technology vendors. Handles ISP issues, coordinates warranty claims, manages software license renewals.
Fail: MSP only handles their own tools and pushes vendor issues back to you.
9. Proactive Communication
What to Check
When was the last time your MSP reached out to you about something that wasn’t a billing issue? Have they ever recommended a change to your setup that wasn’t an upsell?
What We Found
Many providers operate in a purely reactive mode. They wait for you to report a problem, fix it (eventually), and send an invoice. A good MSP contacts you when they notice a trend: a hard drive showing early signs of failure, a device running out of storage, a software version approaching end-of-life. If the only time you hear from your MSP is when there’s a problem or an invoice, they’re not managing your IT. They’re waiting for it to break.
Pass/Fail
Pass: Regular proactive communication about your environment. Recommendations that benefit you, not just the provider’s revenue.
Fail: Only hear from MSP when there’s a problem or an invoice. No proactive recommendations in the last 6 months.
10. Contract Transparency
What to Check
Pull your contract. Can you clearly identify what’s included, what’s excluded, what triggers additional charges, and how to terminate the agreement?
Pass/Fail
Pass: Clear scope definition, explicit exclusions, defined per-incident triggers, reasonable termination terms, and pricing that matches invoices.
Fail: Vague scope language, undisclosed fees appearing on invoices, or termination terms that make it practically impossible to leave.
Scoring Your MSP
- 9–10 passes: You have a solid provider. Keep them.
- 7–8 passes: Decent but with gaps. Bring the failures to their attention and set a 90-day timeline for improvement.
- 5–6 passes: Below standard. Start shopping for alternatives while pushing for improvements.
- Below 5: You’re paying for managed IT but not receiving it. Start a transition plan.
Red Flags That Don’t Need a Checklist
Some things are obvious signs of a bad MSP relationship. You don’t need a scoring framework for these:
- You report the same issue multiple times and it keeps coming back.
- You can’t reach anyone when there’s an emergency.
- Your invoices contain charges you weren’t told about in advance.
- The provider resists giving you access to your own passwords or documentation.
- They blame your equipment for every problem without investigating.
- They sell you new hardware or software to fix issues that proper configuration would solve.
- You’ve never received a report showing what they’ve done for you.
If any of these sound familiar, you don’t need an audit. You need a new provider.
Frequently Asked Questions
How often should I evaluate my managed service provider?
At minimum, once per year before your contract renewal date. A full evaluation using this checklist takes a few hours and should be done 90 to 120 days before renewal so you have time to address gaps or find an alternative. If you’re having frequent issues, run the evaluation sooner. You can also use individual checklist items on an ongoing basis. For example, request a patch compliance report quarterly and a backup verification log every time you receive your monthly report.
What is the biggest red flag when evaluating an MSP?
Untested backups. If your MSP cannot produce documentation showing a verified restore test within the last six months, your disaster recovery plan is unproven. Every other service your MSP provides protects your business during normal operations. Backup is the service that protects your business when everything else fails. A backup that has never been tested is not a backup. It is a file that might or might not contain your data in a format that might or might not be recoverable. This was the most consistent failure we found across years of auditing MSP contracts.
Can I run this evaluation on my own MSP without their cooperation?
Partially. You can check your own ticket history, review your invoices, verify whether you’ve received reports, and test whether you can reach support during and after hours. But some items, like patch compliance and monitoring coverage, require the MSP to provide data. If they refuse to share basic operational reports about your own environment, that tells you everything you need to know about their transparency and accountability. A good provider will welcome the review because it gives them a chance to show their work.
What should I do if my MSP fails multiple checklist items?
Document the specific failures and present them to your MSP’s account manager. Give them a concrete timeline, usually 60 to 90 days, to bring the failing items to pass. Request written acknowledgment of the issues and a remediation plan with milestones. If they meet the timeline, continue the relationship. If they don’t, or if they push back on the evaluation itself, start contacting alternative providers. Do not wait for your contract to expire to begin shopping. It takes 30 to 60 days to evaluate and onboard a new MSP, and you need overlap with your cancellation notice period.
How does Mr. Fix IT Geeks handle these audit points?
We built our service around this exact checklist. Every client gets 24/7 monitoring with automated agent-status checking, monthly patch compliance reports, quarterly verified backup restores with documentation, active security alert monitoring, defined SLA response times, maintained documentation including network diagrams and password vaults, and a monthly health report that covers all of these items in plain English. We audit ourselves using the same framework we used to audit other providers. The monthly report is the proof.