• In Cohorts I and II, patients must have R/R AML and MDS (venetoclax naïve or venetoclax exposed)

‣ Relapsed AML is defined as the appearance of 5% or greater myeloblasts in the bone marrow or peripheral blood after achieving a complete remission (CR), CR with partial hematologic recovery (CRh), or CR with incomplete hematologic recovery (CRi). Patients with mutations in FLT3, IDH1 or IDH2 must have failed or been intolerant of a Food and Drug Administration (FDA) approved FLT3, IDH1 or IDH2 inhibitor before enrolling on study

⁃ Refractory AML is defined as failure to achieve a CR, CRh, or CRi after one of the following regimens: (i) Two cycles of intensive induction chemotherapy with a cytarabine containing regimen (e.g., 7+3, mitoxantrone, etoposide, cytarabine \[MEC\], high-dose cytarabine \[HIDAC\], etc.) or, (ii) Two cycles of hypomethylating agent (HMA)/venetoclax or low-dose cytarabine (LDAC)/glasdegib or, (iii) Four cycles of HMA monotherapy

⁃ Relapsed MDS is defined as: (i) Intermediate, high, or very high-risk disease by International Prognostic Scoring System-Revised (IPSS-R) and, (ii) Any relapse after achieving any 2023 IWG MDS defined response

⁃ Refractory MDS is defined as: (i) Intermediate, high, or very high-risk disease by IPSS-R and \> 5% blasts in the bone marrow or peripheral blood, (ii) Failure to achieve a response (as per IWG 2006 criteria) after four cycles of HMA monotherapy, or (iii) Two cycles of HMA + venetoclax

• In Cohort III, patients must have prior untreated high-risk MDS

‣ MDS with \> 5% blasts in the bone marrow or peripheral blood AND

⁃ IPSS-R high or very high-risk disease OR

⁃ Molecular International Prognostic Scoring System (IPSS-M) high or very high-risk disease

⁃ No prior use of deoxyribonucleic acid methyltransferase inhibitor (DNMTi) therapy with the exception of one single cycle of DNMTi. Prior use of erythropoiesis stimulating agents (ESA), thrombopoietin agonists, lenalidomide, and luspatercept are allowed

• Age ≥ 18 years. Because no dosing or adverse event data are currently available on the use of SM08502 (cirtuvivint) in combination with ASTX727 in patients \< 18 years of age, children are excluded from this study

• Eastern Cooperative Oncology Group (ECOG) performance status ≤ 3

• Total bilirubin ≤ 2 x institutional upper limit of normal (ULN) (unless bilirubin rise is due to Gilbert's syndrome or of non-hepatic origin; in that case a cut off of ≤ 4 × institutional ULN will be used)

• Aspartate aminotransferase (AST)(serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT)(serum glutamic pyruvic transaminase \[SGPT\]) ≤ 3 x institutional ULN (unless considered due to organ involvement by the patient's myeloid malignancy; in that case a cut off of ≤ 5 x institutional ULN will be used)

• Glomerular filtration rate (GFR) ≥ 45 mL/min/1.73m\^2

• If female, patient must be either:

‣ Postmenopausal (surgically sterile or age \> 55 years with no menses for 12 or more months without an alternative medical cause or age equal to 55 or less with no menses for 12 or more months without an alternative medical cause and a follicle stimulating hormone \[FSH\] level \> 40 IU/L); or

⁃ Of children bearing potential. These patients must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Patient must agree to have a negative urine or serum beta-human chorionic gonadotropin (HCG) test result during screening and repeated within 7 days prior to study drug (local labs are allowed) to be eligible

• The effects of SM08502 (cirtuvivint) and ASTX727 on the developing human fetus are unknown. For this reason, and because these agents are known to be teratogenic, women of child-bearing potential must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and during the treatment therapy. Women of childbearing age should agree to use adequate contraception for 7 months after completion of SM08502 (cirtuvivint) administration. For ASTX727, adequate contraception must continue for at least 6 months after last dose of ASTX727. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study and for the duration of the study and at least 4 months after the last dose of SM08502 (cirtuvivint) and 3 months after last dose of ASTX727. Women who are lactating must refrain from breastfeed during the study and at least for two weeks after last dose of ASTX727

• Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial

• HIV-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial

• Ability to understand and the willingness to sign a written informed consent document. Legally authorized representatives may sign and give informed consent on behalf of study participants