• Diagnosis of 1) AML (World Health Organization \[WHO\] classification definition of \>= 20% blasts, excluding acute promyelocytic leukemia \[APL\]), or 2) high risk MDS (\> 10% bone marrow blasts)

• For frontline cohort: Patients aged \>= 60 years old

• For relapsed or refractory cohort: Patients aged \>= 18 years old

• For frontline cohort: Patients must be chemonaive, i.e., not have received any chemotherapy (except hydrea or 1-2 doses of ara-C for transient control of hyperleukocytosis) for AML or MDS. They may have received transfusions, hematopoietic growth factors or vitamins for an antecedent hematological disorder (AHD) or for AML. Temporary prior measures such as apheresis, ATRA (all-trans retinoic acid), steroids or hydrea while diagnostic work-up is being performed are allowed and not counted as a prior salvage. Supportive care therapy for MDS (growth factors, transfusions) will not be considered as prior therapy for MDS/AML and these patients will be enrolled to the frontline cohort of the study if they are otherwise eligible

• For relapsed or refractory cohort: Patients who have received at least one prior therapy for AML or for MDS (with \> 10%) blasts will be eligible. Patients may have received up to 4 salvage regimens for AML and/or MDS (defined by the International Prognostic Scoring System \[IPSS\] classification). Patients who receive MDS directed therapies considered not purely supportive such as hypomethylating agents (HMAs), lenalidomide, investigational therapies, will be enrolled to the salvage cohort if they are otherwise eligible

• In the absence of rapidly progressing disease, the interval from prior treatment to time of initiation of protocol therapy will be at least 2 weeks for cytotoxic agents or at least 5 half-lives for cytotoxic/noncytotoxic agents (whichever is shorter). The half-life for the therapy in question will be based on published pharmacokinetic literature (abstracts, manuscripts, investigator brochure's, or drug-administration manuals) and will be documented in the protocol eligibility document. The use of chemotherapeutic or anti-leukemic agents is not permitted during the study with the following exceptions: (1) intrathecal (IT) therapy for patients with controlled central nervous system (CNS) leukemia at the discretion of the principal investigator (PI). (2) Use of cytarabine (up to 2 g/m\^2) or hydroxyurea for patients with rapidly proliferative disease is allowed before the start of study therapy and for the first four weeks on therapy. These medications will be recorded in the case-report form

• Eastern Cooperative Oncology Group (ECOG) performance status =\< 2

• Serum biochemical values with the following limits unless considered due to leukemia

• Creatinine \< 1.8 mg/dl

• Total bilirubin \< 1.8 mg/dL, unless increase is due to hemolysis or congenital disorder

• Transaminases (serum glutamate pyruvate transaminase \[SGPT\]) \< 2.5x upper limit of normal (ULN)

• Potassium, magnesium, and calcium (normalized for albumin) levels should be within institutional normal limits

• Ability to take oral medication

• Ability to understand and provide signed informed consent

• Baseline left ventricular ejection fraction by echocardiogram (ECHO) or multigated acquisition scan (MUGA) \>= 50%

• Women of childbearing potential (WOCBP) must have a negative serum or urine pregnancy test within 7 days. Men must agree not to father a child and agree to use a condom if his partner is of child bearing potential

• WOCBP must use appropriate method(s) of contraception. WOCBP should use an adequate method to avoid pregnancy until at 30 days after the last dose of investigational drug. Women who are not of childbearing potential (ie, who are postmenopausal or surgically sterile) as well as men with azoospermia do not require contraception. Appropriate methods of birth control include: birth control pills, condoms, intrauterine device (IUD), or other Food and Drug Administration (FDA) approved birth control methods

• Patients may be concurrently enrolling in supportive care clinical trials. Other investigational agents that are used for treatment of other cancers will not be allowed