• Prior histopathologically proven diagnosis of astrocytoma (grade 2-4) or oligodendroglioma (grade 2-3) according to the World Health Organization (WHO) 2021 Classification System that is progressive or recurrent following at least one prior alkylating chemotherapy regimen (i.e., temozolomide and/or lomustine), +/- radiation therapy

• Patient's tumor must have a known mutation in IDH1 or IDH2. IDH1/2 mutation status must be confirmed by DNA sequencing and could have been performed in any CLIA/CAP-certified laboratory. IDH1/2 mutational testing could have been performed on patient's tumor either at initial diagnosis or on a subsequent recurrent tumor.

• Safety Run-In and Phase 2 (Arm A and Arm B) patients:

‣ All Safety Run-In and Phase 2 patients: patients with any contrast-enhancing tumor must have measurable disease per RANO criteria (defined by at least 1cmx1cm of contrast-enhancing tumor). Patients with exclusively non-enhancing tumors must have least a 25% increase in bi-dimensional product of FLAIR signal abnormality (measurable disease) compared to the patient's best MRI scan (smallest bi-dimensional product of FLAIR signal abnormality) obtained following completion of the patient's most recent line of therapy

⁃ Safety Run-In: Must have failed temozolomide OR another alkylator (e.g. lomustine, procarbazine, carmustine). May have failed an unlimited number of prior systemic regimens, +/- prior radiotherapy.

⁃ Arm A: Must have failed temozolomide AND another alkylator (e.g. lomustine, procarbazine, carmustine). May have failed an unlimited number of prior systemic regimens, +/- prior radiotherapy.

⁃ Arm B: Must have failed temozolomide OR another alkylator (maximum one prior chemotherapy regimen) +/- prior radiotherapy, AND must have gone at least 12 months since last treatment (chemotherapy or radiotherapy).

• Surgical patients (Arm C and Arm D):

‣ Must have clinical indication for surgical resection of the suspected recurrent/progressive tumor, as determined by patient's care providers; measurable disease is not required

⁃ 5-aminolevulinic acid (5-ALA) is not allowed for intraoperative tumor visualization due to the photosensitizing agent interaction with ATRA

⁃ Patient may have had an unlimited number of relapses and prior therapy regimens

• Patient must have documented 1p/19q and O-6-methylguanine-deoxyribonucleic acid (DNA) methyltransferase (MGMT) methylation testing. If either of these has not been performed previously, they can be done prior to enrollment.

• Patients must be able to undergo MRI of the brain with gadolinium. Patients must be maintained on a stable or decreased dose of corticosteroid regimen (no increase for 5 days) prior to this baseline MRI.

• Patients must have recovered from severe toxicity of prior therapy; the following intervals from previous treatments are required to be eligible:

‣ 12 weeks from completion of radiation

⁃ 6 weeks from a nitrosourea cytotoxic chemotherapy

⁃ 3 weeks from a non-nitrosourea cytotoxic chemotherapy

⁃ 4 weeks from any investigational (not Food and Drug Administration \[FDA\]-approved) agents, or within a time interval less than at least 5 half-lives of the investigational agent, whichever is shorter

⁃ 2 weeks from administration of a non-cytotoxic, FDA-approved agent (e.g. abemaciclib, olaparib, etc)

• If patient is on systemic corticosteroids to treat brain edema and/or brain edema-related symptoms, the dose must be 2mg of dexamethasone (or equivalent) daily or less for a minimum of 5 days prior to first dose of retifanlimab.

• Patients must be able to swallow oral medications

⁃ Age 18 or older

⁃ Karnofsky performance status greater than or equal to 60

⁃ Life expectancy \>3 months

⁃ Adequate organ and marrow function:

∙ Total bilirubin \<1.5 x upper limit of normal (ULN) (except patients with suspected Gilbert's Syndrome, who are eligible for the study but exempt from the total bilirubin eligibility criterion)

‣ ALT and AST ≤ 2.5x ULN

‣ Calculated CrCl ≥ 30 ml/min (glomerular filtration rate can also be used in place of CrCl)

‣ Absolute Neutrophil count ≥1,500/uL

‣ Platelets ≥ 100,000/uL

‣ Hemoglobin ≥ 9 g/dL

⁃ Reproductive Status

• Women of childbearing potential (WOCBP) must have a negative serum pregnancy test (minimum sensitivity 25 IU/L or equivalent units of HCG) within 7 days prior to the start of study drug.

∙ b) Women must agree to not breastfeed during the study or for 180 days after the last dose of study treatment

∙ WOCBP must agree to use an adequate method to avoid pregnancy (as defined below) from the time of study screening through 180 days from last dose of study drug

∙ Males who are sexually active with WOCBP must agree to follow instructions for method(s) of contraception (as defined below) starting with the first dose of study drug through 180 days after the last dose of study

∙ Azoospermic males and WOCBP who are continuously not heterosexually active are exempt from contraceptive requirements. However, these WOCBP must still undergo pregnancy testing as described in this section.

⁃ At a minimum, participants of childbearing potential who are sexually active and their partners must agree to the use of a highly effective form of contraception (as defined below) throughout their participation beginning with the time of consent, during the study treatment, and for 180 days after last dose of study treatment(s).

⁃ HIGHLY EFFECTIVE METHODS OF CONTRACEPTION:

‣ Hormonal methods of contraception including combined oral contraceptive pills, vaginal ring, injectables, implants and intrauterine devices (IUDs) by WOCBP subject or male subject's WOCBP partner. Female partners of male subjects participating in the study may use hormone-based contraceptives as one of the acceptable methods of contraception since they will not be receiving study drug

‣ Nonhormonal IUDs

‣ Bilateral Tubal ligation

‣ Vasectomy

‣ Sexual Abstinence

⁃ It is not necessary to use any other method of contraception when complete abstinence is elected.

• WOCBP participants who choose complete abstinence must continue to have pregnancy tests.

• Acceptable alternate methods of highly effective contraception must be discussed in the event that the WOCBP participants chooses to forego complete abstinence.

⁃ Participant must, in the opinion of the Investigator, be able to comply with study procedures

⁃ Patients must be able to understand the study procedures and agree to participate in the study by providing written informed consent (or have legally authorized representative sign on patient's behalf if patient physically unable to sign consent due to neurologic deficit)