Writing an assessment

An assessment is the exam behind a certified program. Somebody sits it, the platform scores it, and a pass issues a credential with your name or Revquix's on it. Everything you write lives under assessments.

Sections are pools, not chapters#

This is the one idea the rest of the screen depends on.

A paper is made of sections. A section holds a set of questions and draws a few of them per candidate: you might write twenty and set the section to draw five. Two people sitting the same assessment therefore see different questions.

A retake is drawn from the questions that candidate has not seen yet, so a section needs enough questions for every attempt: draw count x attempts. If a section draws five and candidates get two attempts, it needs at least ten. The checklist stops you sending anything smaller, because with fewer the second paper would repeat part of the first.

Difficulty and the mix#

You can rate each question Easy, Medium or Hard. A question you leave unrated counts as medium.

A section can also set a difficulty mix - say 2 easy, 2 medium and 1 hard out of a draw of five - so every candidate gets a paper of the same shape rather than whatever the draw happened to pick. The mix has to add up to the draw count, and each band then needs its own share of the pool: its count x attempts. With the mix above and two attempts, that is at least 4 easy, 4 medium and 2 hard. The section card tells you as you type if a band is short.

Every candidate still sits a paper worth the same. A section is worth draw count x marks per question, and the assessment total is the sum over its sections - so the total is fixed by the settings, never by which questions came up.

Draw count and marks per question are each 1 to 100. If a section draws more questions than its pool holds, the checklist stops you before it can be sent.

A section's introduction#

Each section can carry a short introduction, up to 600 characters. Candidates read it on a card before the section's first question - what the section is about, or anything they should know before it starts. It is never the place for a hint: anything written there is shown to every candidate.

Section time#

A section can have its own time, set in the Time box beside its draw and marks. Give every section a time, or none: the checklist will not publish a paper where only some sections are timed, because "whatever time is left" is not a limit anybody can be told in advance.

On a timed paper:

• The duration is the section times added up. The Time allowed box becomes read-only and follows the sections as you change them.
• Candidates sit the sections in order, one at a time, with no way back. When a section's time is up the next one opens.
• A candidate can move on early, and the time they did not use carries over to the next section.
• Agreed extra time stretches every section by the same share.

Use it when a later section would give away an earlier one, or when each part deserves its own share of the time rather than whatever is left. A section time is 1 to 360 minutes, the paper as a whole still runs 5 to 360, and like the duration it is frozen once the paper is published. A revision keeps the section times.

What you can ask#

Four question types:

• Single answer - one correct option. At least two options, exactly one correct.
• Select many - several correct options. At least three options, at least two correct, and never all of them.
• Exact answer - a number or a short word the candidate types.
• Coding - the candidate writes a function, and it is marked on the hidden cases of a coding problem you wrote. See Coding questions below.

Essays are no longer offered. An essay needs a person to mark it before the candidate sees a result, and that wait is what candidates like least. An essay already in one of your pools still opens so you can change its type or retire it, and it has to be gone before the assessment is sent for review again.

Coding questions#

A coding question asks one of your own exam problems. The candidate reads its statement and worked examples, writes the function in one of the languages you offer, and presses Run as often as they like before the deadline. Their last run is what counts.

Only your own problems. A question can ask a problem you wrote and no one else's. The picker only shows yours, and the platform refuses any other.

Partial credit is off by default: full marks only when every hidden case passes. Switched on, the mark is the share of hidden cases passed, rounded down. The samples printed on the paper never count toward it.

If our code runner fails on a candidate's code, that is ours, not theirs. The code is kept and run again automatically after they hand in, with a growing wait between tries. You are asked to mark it by hand only if the runner keeps failing, and the grading screen says which case you are looking at.

Open listings. Assessments with coding questions are sold through your Certified Program for now. They cannot be opened to the public until the code runner has been load-tested at three times the expected peak.

The pass bar, the clock and the strike limit#

Pass percent is 1 to 100, and the marks needed are worked out from it and rounded up - a 70% bar on a 3-mark paper needs 3, not 2. Duration is 5 to 360 minutes; on a paper with section times it is their sum and is not set here (see Section time).

Integrity strikes is how many integrity signals a sitting may collect before it is closed early and submitted as it stands. Leaving the exam tab, dropping out of full screen and pasting into an answer all count; signals are recorded whatever this is set to. It is 0 to 20 and starts at 3, and 0 turns the ladder off - signals are still recorded and none of them count, which is a real setting for a low-stakes paper rather than a mistake. When a sitting is closed this way, the reason travels onto the record an employer can see.

All three are frozen once anything has been graded against them. Moving a bar under an attempt that has already been scored would change a result somebody was told.

Attempts and the wait between them#

Attempts is how many times one person may sit the assessment, counting the first: 1 to 5, and 2 to begin with. Retake wait is how many days a failed attempt must wait before the next one opens: 1 to 30, and 7 to begin with. The wait is counted from the moment the failed result is released, and the candidate is shown the date and time the next attempt opens.

After the last attempt, a candidate who has not passed cannot sit this assessment again. Candidates are told how many attempts they have before they start the first one.

Both settings are part of what the reviewer approves, because they decide how deep your pool has to be, and both freeze once the assessment is published.

What this assessment tests#

Every assessment can carry an overview: the page a candidate reads before deciding to sit it. It says what the paper tests, who it is for, what they should know first, what it does not cover, and how to prepare, with an optional sample question in the same style.

Practice questions#

You can add up to ten practice questions. They appear on the assessment's public page with their answers, so a candidate can try the style before the clock starts. They must not be questions from your pool: one that repeats a pool question, or is too close to one in meaning, is refused.

Checking your questions#

Question quality in the editor gives two kinds of advice. Checks are worked out from the paper every time you open it: the right answer being the longest option (candidates learn to pick it), two questions that read almost the same, a skill with too few questions, and roughly how long one paper takes against the time you allow.

Check with AI goes further. The AI solves each question without being told your answer, and Revquix compares its answer with yours. When it picks something else, check the key first - a wrong key fails candidates who were right. It also looks for the usual item-writing slips: a question that can be read two ways, an option nobody would choose, "all of the above", or a question that tests a memorised fact rather than the skill.

None of it blocks anything. Apply opens the question with the suggestion filled in, and nothing changes until you save it. Dismiss asks why; the reviewer reads your reason.

Growing the pool#

A retake is drawn from questions the candidate has not seen, so a pool needs more questions than it draws. Each section says what it is short of, and offers ways to get there faster than writing every question by hand:

• Make variants (the spark on a question) drafts new questions that test the same idea with different numbers or a different situation. Variants are kept as a set: a paper never holds two of them, and a retake may draw one.
• Draft questions with AI writes new questions for a difficulty and a skill the section is short of.
• Import questions reads questions pasted from your own document, or a CSV file. The CSV is read with no AI at all, and the AI never guesses an answer your text does not give.

Everything comes back as a draft. Nothing joins the pool until you open it, check it - the answer above all - and add it. The question then records whether it was drafted with AI, and a reviewer sees the mix. What you add is yours to stand behind, like anything you write.

Opening it to more people#

An assessment starts out reachable only through the programmes you attach it to. You can ask to open it further:

• Anyone with the link - it gets a public page you can share, and is not in the catalogue.
• The public catalogue - for Revquix-issued assessments and mentors Revquix invites. A reviewer may open it as a link instead.

A reviewer decides, and before they can approve it the assessment has to be published, have an approved overview, and have a deeper pool: anyone can reach an open paper, so every section needs enough questions for three sittings without a repeat, or your attempts if that is more. The panel in the editor lists whatever is still missing before you ask.

A pass through the open route says so on the certificate and its public page, and a pass through your programme says that, with your name - so opening an assessment never makes your programme's credential look like the free one. Once approved, the public address never changes, even across revisions. Withdrawing keeps it reserved for you.

Selling sittings#

Once a reviewer has opened your assessment, you can sell sittings of it instead of leaving it free. Create a service of the Assessment sitting type and choose the assessment in its The assessment section. It is priced, discounted and paid out like your other services, with no session to run and no calendar needed.

• One purchase is one sitting. Each attempt the assessment allows is bought separately, and a buyer can hold one unused sitting at a time.
• Refundable in full until the buyer starts it, and not after - a failed sitting is not refunded. The buyer is told this before they pay.
• Your earnings are held until the sitting's result is released, then paid like any other sale. A sitting nobody uses is paid to you when it expires, a year after purchase.
• If Revquix voids a sitting because something went wrong on our side, the buyer gets the sitting back. You are not paid twice for it.
• While a sitting service for an assessment is live, starting that assessment on the open route needs a sitting. Pausing the service stops sales without making the assessment free; archiving it does.
• Once someone has bought a sitting, the assessment the service sells is fixed. To sell a different one, create a new sitting service. A revision of the same assessment needs nothing: sittings follow it.

An assessment Revquix issues is sold for credits at a price Revquix sets, not as a mentor service.

You write it, a reviewer publishes it#

There is no publish button on this screen, and that is the design rather than a missing feature.

You will also see who the credential is issued as - you, hosted and verified by Revquix, or Revquix itself. That is set during review and is read-only to you.

Reading the certificate before you send it#

The editor draws the certificate your assessment will issue, beside the fields that decide what is on it: the title in the serif it will be printed in, your name as Course Mentor, and the score a candidate who just cleared your bar would carry.

It is worth a look before you send, because the title is the one thing on that sheet you control and it is read with none of this page around it. The pass bar, the time allowed and any integrity note are deliberately not on the certificate - they live on the verification page the code opens, which is the record an employer checks.

After it is published#

The structure freezes. Sections, draw counts, marks, the pass bar, the attempts, the retake wait, each section's difficulty mix and each question's difficulty stop being editable, because they are what already-graded attempts were measured against.

Two things still move:

• Wording. Prompts and explanations stay editable, so a typo or an ambiguous sentence can be fixed without a new version.
• The pool. You can keep adding questions. A new question waits for a reviewer to stamp it and is not drawn until then - so adding one and seeing nothing change is the system working, not a failure.

For anything structural, press Revise. That copies the assessment into a draft you can edit freely while the published one stays live and sellable, and the copy replaces it when approved.

Archive stops new sittings for good. A published assessment that a Certified Program sells, or that has an open listing, cannot be archived: its buyers would be left with an exam they paid for and cannot sit. Revise it instead - approving the revision moves the program to the new version - or take it off the program or the listing first.

How it is doing#

Once candidates have sat it, Analytics on the assessment shows what their results say about the paper: how many sat it and passed, the spread of scores, results by skill, and, for each question, how it performed. Every revision of the assessment counts, because a revision is the same exam.

A question's numbers appear once 30 candidates have answered it. Below that one person moves a percentage by three points, so the page says how many more it needs instead of showing a number that means nothing.

• Facility is the share of the question's marks candidates earned.
• Discrimination is how much better the strongest candidates, judged on the rest of the paper, did on it than the weakest. Below 0.2 it says little about skill. Clearly below zero, stronger candidates do worse on it, which almost always means the key is wrong or the question reads two ways.
• Who chose what shows each option's share overall and in the top and bottom groups. A wrong option the strong candidates prefer to your key is the plainest sign of a wrong key; an option nobody picks does no work.

Questions with something to look at come first, with the reason in words: Check the key, May have leaked (it suddenly got much easier this week), Over-exposed (on most papers of an open assessment), a difficulty label far from how candidates found it, and more. They are advice and never block anything. Explain asks AI to put one question's numbers into words; it is sent the question, its options and your key, and the numbers - nothing about any one candidate. The statistics are recomputed every night, or now with Recompute now.

Revquix reviewers see the same numbers, and are told when a question looks mis-keyed or leaked, or an open assessment looks like it is being harvested.

Marking and appeals#

Written answers and appeals land on your grading page. A candidate can appeal a result once, within 14 days of it being released, and you are notified when they do. Answer within 7 days: after that the appeal passes to Revquix reviewers, who can re-mark it and answer it themselves, and the candidate is told. You can still answer it after that; whoever answers first closes it. If Revquix answers it, you are told what was decided and why.

Attaching it to what you sell#

An assessment does nothing on its own. It is attached from your service or package editor, which is why the row sits under Sell rather than beside grading - see your services.

The package shows the assessment's title as what the certificate will say, and its attempts as what the buyer gets. Neither is typed on the package: to change them, change the assessment, and an approved revision carries the new values onto every package attached to it.

Once somebody has bought the package, its certificate terms are fixed: whether it offers a certificate, which assessment, and how many sessions come before the exam. Those are what the buyer paid for. To change the assessment, revise it; for anything else, create a new package. Offering a certificate on a package that sold without one is allowed.