Lab Essentials

About this episode

‘Fusing AI with Certification and Labs’ explores how Kinaxis transformed its training and certification programs to support rapid growth by focusing on scale, quality, and velocity. Through partnerships with CloudShare and Criterion, the company shifted to hands-on, role-based learning and secure, AI-driven assessments that validate real-world skills, improve learner engagement, and increase the overall value of certification.

Transcript of the video

Lee Berkman:
Okay, everyone, it is official. I can already see participants joining. Hi, everyone. Thanks very much for making the time for our webinar today and for joining our panel of experts. It is always nice to see CloudShare clients on the call, team members, and many familiar names. We would love to engage with everyone as much as possible, and even more so, we would love for you to engage with us. Please share where you are dialing in from today, a little about your role, and what you do at your company. Feel free to share company names. We are all here to learn from one another, and that is exactly why we have a number of experts joining us to talk about how they have addressed real world goals and problems using the right tools. We are seeing people joining from San Francisco, Tel Aviv, Toronto, Germany, Minneapolis, Boise, Mexico, Austria, India, and Edinburgh. It is great to see such a global audience. We will give everyone a few more minutes to join, and then we will kick off. For those already here, my name is Lee Berkman. I am from CloudShare and part of the pre sales team, and I enjoy supporting these webinar events. I am fortunate enough to work with Carmit and Fayez, and Chad is a joint partner with CloudShare. We will discuss that relationship in more detail during the call. Please ask questions throughout the webinar. Please give us feedback and share your opinions. Feel free to use the Q and A if you have specific questions about technology, processes, or anything else you would like us to address either during the webinar, when appropriate, or at the end during Q and A. It is obvious that you all know how to use the chat, so thank you very much for engaging with us and joining from all different time zones. Carmit, Fayez, Chad, give me some nods or thumbs up if we are ready to go. Great. Without further ado, I would like to formally welcome everyone. We are actually repeating a very successful session that we first presented in person at a Sedma event in the UK. Today we are going to discuss how Fayez had a vision for his training programs and how he partnered with the right vendors to make sure that vision could happen. Fayez is the VP of Training and Enablement at Kinaxis. He is an expert in this space and has also worked with CloudShare and with me at a previous organization as a sales leader. Fayez is also the chairman of Sedma, so I will let him give a shout out to Sedma shortly. We also have Chad Fletcher, Director of Product at Criterion, who brings unique cybersecurity and user identification capabilities, and Carmit Pelleg, our VP of Product at CloudShare, who will show some of the capabilities that empower Fayez and his end users to get hands on with technology. We are going to introduce the Kinaxis problem statement, go through the CloudShare tool set and functionality that Fayez introduced and adopted, and then show where our partnership with Criterion came in and what unique solutions they bring. Fayez, I think I have done enough talking for now. Welcome. We would love a quick introduction to what Kinaxis does, a little more about your role, and your position in the marketplace.

Faez Ahmed:
Thank you so much, Lee, and good morning, good afternoon, and good evening wherever you are. Welcome to this session. As Lee mentioned, I run the training and enablement program at Kinaxis. I want to start by saying that Kinaxis has the most amazing, caring, and intelligent people working for it. The type of people in your company matters just as much as the product, if not more. In fact, during my interview at Kinaxis, our Chief Customer Officer, Mike Major, and our VP of HR, Stephanie Hosick, were interested in who I was as a person and whether I was a culture fit. It is too late for them now to decide whether that was the right hiring choice. Kinaxis is a supply chain management company headquartered in Canada, with a global presence. Our AI powered concurrent planning platform enables companies to connect data, processes, and people, allowing them to respond to key questions in a matter of seconds. It is a great product, and I would say the best in the supply chain world. This also speaks to our ability to execute. Kinaxis has been positioned highest for the tenth consecutive time, and that comes from both an exceptional product and exceptional people. What is most important for today is the context of where we were about two years ago and how much has changed since then. When I joined, things were already working well. We had strong partner engagement, courses aligned closely to our product capabilities, excellent uptake on training subscriptions through sales and professional services, great custom training, more than forty certifications, and close to twenty thousand certified people in our ecosystem. At that point I had to ask myself why I had been hired if things were going so well. The answer was that Kinaxis was preparing for hypergrowth and wanted to make sure that what existed in training and enablement was future proof. That was the key consideration in the discussions I had after the interview. The question was whether I could come in, assess what we had, and determine whether we were ready for where the company was going to be in a few years.

Lee Berkman:
Let us engage everyone with a quick poll. What would you do in that situation. Hire Fayez, ask ChatGPT what to do, review your vendor tech stack, build new content, or something else. Fayez, I already have some feedback coming in and it is pretty split. Before I share the results, what do you expect will be the most popular answer.

Faez Ahmed:
It has to be the hairstyle. Why else would people want to hire me.

Lee Berkman:
We have the results, and the majority of people said they would review their vendor tech stack, which works very well with where your plans and vision went. Some people also chose ChatGPT and hiring Fayez. All of the above appeared in the chat as well.

Faez Ahmed:
That is fair. Thank you to everyone who said hire Fayez. I appreciate it. The tech stack is the key thing, and I will walk you through the kind of things we had to do to get where we are today. I recently presented this to an excellent audience of education leaders, managers, and collaborators at the Sedma Enterprise Conference in London. They came up with many different ways of addressing this challenge. By the way, if you have not heard of Sedma Enterprise, look it up. If you work in education and do not know Sedma, you are missing out on a huge amount of best practices in training, education, and certification. Our approach was built on two basic principles. We already had almost twenty thousand certified people in our ecosystem who had gone through the typical high stakes and low stakes certifications that most companies use. The question I had to ask myself was how I could know whether those people had the right competencies, not just when they took the assessment, but today and tomorrow. In a world where skills become redundant in days, continuous assessment is not just an option, it is professional survival. That was a key idea for us. I also love the Benjamin Franklin quote, because it supports our decision to transition our curriculum, our learning management system, our certification program, our labs, and our infrastructure toward a more hands on approach. I asked myself whether certification truly linked to continuous learning while still validating competence. I strongly believe certification is more than a piece of paper, or these days a digital badge. It is a passport to transformation and a bridge between who you are and who you aspire to be. As education leaders, we need to make that vision real, but it must have value for both the business and the individual taking the certification. To bring that vision to life, I needed the right partners, and that is where CloudShare and Criterion came in after a relentless search for vendors that truly fit the bill.

Lee Berkman:
And how broad is the audience you are certifying. Are we talking about partners, resellers, customers, and internal teams. Who is actually receiving this service.

Faez Ahmed:
It is the entire ecosystem of internal teams, partner consultants, and customers. There is a difference between the types of certifications for internal teams and partners versus customers, but for our internal teams and partners, I needed to make sure they understood what they had learned and were able to apply it. That was a key consideration in selecting a certification vendor and a lab provider that could all work together in one integrated model.

Lee Berkman:
And that takes us to the three guiding principles you put in place.

Faez Ahmed:
Exactly. In order to address Kinaxis growth, we adopted three guiding principles: scale, quality, and velocity. The key to unlocking our ecosystem’s potential was not just talent, but strategic skills development. We knew we needed to replace our learning management system because it did not fit our ambitions to scale. We moved away from our internal infrastructure and trusted CloudShare to help us scale faster. For curriculum, we introduced tools like Synthesia and used AI to completely revamp our partner enablement programs around a more hands on approach. We also replaced our previous certification vendor and entrusted that vision to Criterion. All of this was part of the broader transformation. What you see now is the outcome of that plan. It has been truly transformational. We do not have time to go through every detail, but the core theme throughout all of it is the hands on approach. Our partner enablement class, for example, is now a practical mini implementation, and we are scaling that through our authorized training partners. As for certification, we moved from a product based approach to a role based performance assessment model that we are about to roll out. It is a complete transformation of where we were into where we are today and where we are going.

Lee Berkman:
One thing that has always interested me is the change from free certifications to paid certifications. Previously people had free access and could simply attend an event, take a certification, and earn it. This was the first time you introduced paid certifications. I am curious about the reception and adoption. Did you see a decline, or did you see higher quality. What happened.

Faez Ahmed:
That is always an interesting dilemma in the education space. The key test for us was our main conference event, where certifications had historically been available free of charge and could be taken as many times as people wanted. In my opinion, while that model gave customers and partners access, it devalued the certification itself. So we changed the vision. Every time I have spoken about charging for certification since then, and explained that it would now be hands on and would carry a cost, the response has actually been very positive. We have not implemented it everywhere yet, but wherever we have discussed it, people have been reasonably happy and have said they would pay for it. At our key customer event last year, where everything used to be free, we started charging for the first time. The customers knew there would now be a cost, whereas in previous years, attending the conference and its learning opportunities carried no charge. We ended up with a waiting list because customers understood the value they were going to get from the new curriculum, the new programs, and the new labs we launched. The classes were full, people were desperate to get in, and we could have added more classes and still filled them. For us, that was a huge indicator that people will pay if the value is there and if the assessment engine is right.

Lee Berkman:
That is fantastic to hear. It also means your training department can demonstrate revenue generation, which always helps when you are asking for more budget and investment. It also reinforces that your certifications mean much more than just a digital badge.

Faez Ahmed:
Absolutely. What we wanted was a seamless path for the learner. They start in our new learning experience platform, the CloudShare labs are integrated into that experience, and Criterion handles the validation and assessment through AI. One of my major motivations was to stop situations where one person would complete work in the labs on behalf of an entire company just to help them reach the next partner tier. I wanted to preserve the sanctity of the certification program. At the same time, the learner experience had to be smooth and cost effective as we scaled. This next view shows how it all fits together. We build the courseware, we build the labs, CloudShare works its magic, and Criterion validates the learner. A key consideration was ensuring that the person taking the lab really was that person before we handed them a credential. Before we move on, I want to call out what amazing partners CloudShare and Criterion have been. Everything I threw at them, they took on board. They tweaked their product roadmaps and addressed our asks quickly. Not once did I hear that something was only on a distant roadmap. I have been so impressed by the collaboration, and I am very grateful for the partnership.

Lee Berkman:
Thank you very much, Fayez. We love working with you too. You mentioned at the beginning the incredible Kinaxis team, and I can vouch for that. We are now going to hand over to Carmit. Before we do, one final poll for the audience. Are you using virtual labs today, and if so, in what form. If not, are you planning to, or does it simply not fit your business. Feel free to add context in the chat as well. Carmit, what do you expect the feedback will be.

Carmit Pelleg:
It will probably tell us who here is already a CloudShare customer and who is still a prospect without a lab solution. That will be very interesting to know, and I will adjust what I show based on the results.

Lee Berkman:
We have the results. The strong majority are already using labs. A very small percentage are planning to, and some are not using them yet. We would love to know why. Carmit, over to you.

Carmit Pelleg:
Thank you, Lee. Following Fayez’s explanation of the Kinaxis training and certification operation from courseware to labs to final assessment, I am going to focus on the lab portion and show you how CloudShare customers leverage AI in labs to scale their enablement operations. For those who are not yet CloudShare customers or lab users, CloudShare is a virtual lab provider. Virtual labs are an essential part of learning and enablement because they are where hands on practice and proof of acquired skills happen. A virtual lab can be standalone or integrated into an organization’s LMS, allowing the participant to access a training environment or predefined playground with a click. The typical flow is that the learner first acquires theoretical knowledge, then clicks a link, and a browser tab opens with a predefined environment that includes everything needed to practice what was just taught, including step by step instructions, resources, exercises, and automated checks. The focus of today’s session is those AI automated checks. Each customer can customize the participant view to match the session context. The main frame displays the platform or any virtual machine required for hands on practice or simulation. Customers can add step by step instructions in sections and, where relevant, add a checkpoint. The participant follows the instructions side by side with the hands on platform. When they complete a section, they click check and immediately receive a result showing whether they got it fully or partially correct. If they hover over the hint, they can see the difference between the expected result and the actual result. In the example we showed, the top field was correct and the bottom field was incorrect. The learner can then fix their work by following the instructions and hints and click check again. In that second attempt, they got the expected result. The key point is that the participant can progress independently without relying entirely on the instructor to confirm whether they are correct. Our visual AI checkpoint can match their work against the expected result. The computer vision algorithm can match complex elements with a very high level of accuracy. It can handle different screen positions, different resolutions, and even partially hidden elements, so the ratio of false positives and false negatives is very low. This has significant value for self paced learning because the participant can be completely independent, guided by the system, and have their work verified automatically. When it is an instructor led session, the instructor receives progress and success information in real time. They can monitor each participant, see where they are in the assignments, prioritize whom to assist, zoom in, take over environments, chat, and share their own environment or the learner’s environment to support them. It is effectively as if the instructor has multiple sets of eyes, one for each participant, all checking and verifying work concurrently. In addition to participant level data, the instructor can see aggregated analytics for the whole session, such as completion rates, pass fail rates on checkpoints, and average time spent per section. To get to that point, the instructional designer first configures everything in the editor, which reflects the participant view. They can preset the content, resources, and sequence of instructions, and where relevant, add a visual AI checkpoint. They capture an element or area on the screen that represents the desired result. For example, the instructional designer may want to verify that two specific fields are updated with exact values. In the participant view, when the learner clicks check, the computer vision algorithm is activated and compares the expected result to the actual one on the participant’s screen. To summarize, our customers leverage AI to scale their enablement operations by extending the instructor’s reach. They can guide participants, verify progress and completion at scale, and open the possibility of shifting more sessions to self paced formats. They can evaluate the progress and success of specific participants as well as assess the entire class in real time. Finally, they can connect completion and success rates to business outcomes such as retention, adoption, and engagement in order to measure impact and return on investment.

Lee Berkman:
Thank you, Carmit. We had a couple of relevant questions. Elizabeth asked about different branding for different customers, and the answer is yes, you can have different branding for different experiences. Lynn asked about analytics, and I want to clarify that the analytics Carmit showed are real time analytics for the instructor during class, but there is also an additional analytics package in CloudShare that lets you view a larger data set beyond a single session. Melinda asked whether guided journeys are saved in an individual environment or a reusable blueprint.

Carmit Pelleg:
By next week they will be saved to the blueprint, so they can be reused for multiple sessions.

Lee Berkman:
Perfect. Chad, you are up next. We are going to ask one final poll question to keep everyone engaged. What are you most excited to learn about today. Chad, before your presentation, can you tell us how you got into this area of AI presence detection.

Chad Fletcher:
I started my career at Criterion thirteen years ago, so I have been with the company a long time. I started in customer success and then moved into product. I am really passionate about finding the balance between a positive candidate experience and maintaining security for an exam or, in this case, labs. We want to make sure the environment is secure and the candidate is who they say they are, but we do not want to stress candidates out further with a complicated process or poor user interface. We want them focused on the lab or test questions, not the proctoring process.

Lee Berkman:
I love that. And with the rise of AI, deepfakes, and all the different ways people can manipulate systems, I imagine this kind of solution is becoming more and more important. Before I hand over to you, do you have an expectation for what people are most interested in learning today.

Chad Fletcher:
If I had to guess, I would say how we balance security with privacy.

Lee Berkman:
The results are almost perfectly split, so people seem interested across the board. Chad, please walk us through it.

Chad Fletcher:
Absolutely. I am excited to share how we are working with Kinaxis to make their labs more secure while also ensuring a better experience for the learners taking the labs. Kinaxis came to us with two main goals. First, they wanted to confirm the learner’s identity and make sure the person taking the lab is who they say they are. Second, they wanted to ensure that the same verified person starts the lab and completes the lab, which is essentially presence detection. Other forms of security were less of a concern for these lab assessments. The focus was identity verification and making sure that same person remained seated for the duration of the lab. Before showing the demo, I want to explain how AI face detection works because hearing the phrase AI can sound daunting. AI does not see photos the same way humans do. It analyzes captured images and looks for facial features such as the shape of the eyes, the size of the nose, the mouth, and the distance between facial elements. It turns all of that into data points and compares those points to other photos or video captures. It then returns a match or confidence score. A high confidence score tells us it is likely the same person. A lower score suggests it might be someone else. From the learner’s perspective, the process consists of three easy steps. First, the user captures a selfie photo. Second, they capture a photo of a government issued ID. Third, when they begin the lab, they allow access to their webcam so we can monitor their video feed and ensure they are present for the lab. The process starts on the desktop, where the learner scans a QR code with their phone or manually enters a code into the mobile check in app. On their phone, they take a selfie and then take a photo of their ID. On the back end, AI compares the selfie with the photo on the ID and also extracts the name from the ID to compare it against the name used to register for the lab. That confirms that the person who registered is indeed the person starting the assessment. Once that is complete, the learner is in the lab on their desktop, and we begin monitoring the video feed through AI. There is no live proctor watching the webcam. The AI is simply checking that the candidate’s face remains in view throughout the lab. If the face is no longer detected, access to the lab is paused and the learner receives an alert that they are not allowed to leave the test area. Once their face comes back into view, they can resume. On the back end, our team can see the AI detecting when the candidate leaves the frame and when they return. That also means the same verified person has to return to the test area and complete the lab. On privacy, which is obviously important, the photos captured during ID check are deleted immediately after they are used. Unlike some other proctoring solutions, we do not save session video. The video is only used in real time by the AI. We also do not store biometric data beyond the lab event. The data is processed in real time, used temporarily, and then deleted once the lab is complete. We specifically designed the process to be GDPR compliant.

Lee Berkman:
We had a question from Justin about situations where labs run over multiple days. Can the AI proctoring solution still work in that model.

Chad Fletcher:
Yes. As long as it is the same lab event, we can continue using the solution over multiple days. A candidate can exit the lab and come back later if needed.

Lee Berkman:Mark asked whether the AI has trouble if someone wears glasses and then takes them off.

Chad Fletcher:
For basic face detection, glasses are generally not an issue. They become more of a challenge only if you are specifically trying to track eye movement, because glare can interfere with that.

Lee Berkman:
Another question was about bandwidth and whether clients in areas with poor internet can still use the solution.

Chad Fletcher:
The processing is done on our end, so the candidate only needs enough bandwidth to sustain a video stream. The requirement is fairly low, so this generally works in most environments.

Lee Berkman:
Fayez, there was also a question for you about the gamification you are incorporating into the learning and automation process and how that ties into the Kinaxis certification strategy.

Faez Ahmed:
That has not yet been implemented, but it is part of our plan going forward. We are going to gamify how the lab environment works. As you progress through a lab, if you make a mistake, we will eventually provide the answer, but in stages. You will receive hints, tips, and tricks. The more hints you need, the more points you lose. That has not been launched yet, but it is where we are going. The whole process has to be automated and validated by CloudShare so that learners get the answer in the end and learn along the way. At the same time, there is learning happening for the curriculum team and the training management team. If a particular task consistently requires too many hints, that tells us the concept is either not in the curriculum or has not been explained well enough, and we need to fix it. So both the learner and the training organization are learning from the process.

Lee Berkman:
That is a great point, and it also reminds everyone that we are only focusing today on a very specific part of the learner journey, namely the hands on portion. There is of course also the theoretical learning, the grading afterward, the quizzes, and all the other parts that fit together across these tools. Heather asked whether labs are being used just for training or also for actual certification exams.

Faez Ahmed:
We use the CloudShare labs for both. They are part of the curriculum and attached to the course. The assessment part is where Criterion comes in. We need to make sure the individual, as Chad explained, is the person doing the assessment. One thing I want to clarify, because I saw some questions on this, is that we do not mind if other people are physically nearby. What we care about is that the person doing the lab is the one actually working in the lab. In real life, if you are on a project and get stuck, you ask colleagues for help. The assessment is based on you doing the work, and that is where the learning really happens.

Lee Berkman:
One more question came in about whether labs can support more open ended evaluation, where the system looks at the learner’s process rather than requiring an exact final output.

Carmit Pelleg:
At the moment, not yet. Right now the system verifies an expected result, but we are actively looking at solving the broader use case of evaluating the steps taken.

Lee Berkman:
And to clarify that further, the current release is designed around exact outputs, and the next phase will provide more variables and more control over how grading works. For anyone who wants to go deeper into that, we would be happy to demonstrate more. We are almost at the top of the hour, so for anything we were not able to address, please reach out. I am part of the customer facing team, and my email is leeb@cloudshare.com. If you have questions about CloudShare, want a direct introduction to Chad for AI identity checking, or want more information from Fayez, we would be happy to connect. There were a lot of thank you messages to our panelists, and I want to echo that. Everyone took time to prepare this presentation, rehearse it, and be here today. Carmit, Chad, Fayez, thank you all for joining. Thank you to everyone who participated and asked such thoughtful questions. It made this session more engaging and more fun. The recording of this session will be shared, so if you would like to rewatch it or share it with colleagues, please do. Enjoy the rest of your evening, everyone.

Faez Ahmed:
Thank you all very much.

Lee Berkman:
See, I told you I was protecting Fayez at all costs.

Faez Ahmed:
I will pay him the fiver later. Thank you all.

Chad Fletcher:
Thank you, everyone.